16 open-source alternatives ranked by real activity

Open-source Amazon SageMaker alternatives

A curated, ranked list of the 16 best open-source alternatives to Amazon SageMaker.

The best open-source alternative to Amazon SageMaker is vLLM. If that doesn't suit you, other good options are SGLang, Airflow, Ray and MLflow.

Commercial alternatives to Amazon SageMaker include Hugging Face, Lightning AI, Azure AI Foundry and Baseten. See all 25.

Amazon SageMaker alternatives are mainly AI infrastructure tools, but some are also data pipeline & ETL tools and AI tools. 12 of them shipped code in the last 30 days, 16 can be self-hosted, and 14 use a permissive licence.

Last updated November 13, 2026 · ranked by GitHub stars, growth and recent commits

vLLM

A Python library and server for fast, memory-efficient LLM inference and serving, with PagedAttention, continuous batching and broad quantization support.

GitHub stars
97k
Last commit
yesterday
Latest release
v0.33.0rc3
Licence
Apache-2.0
Self-hosted
Yes
vllm.aivLLM homepage screenshot

vLLM is an open-source library for running and serving large language models efficiently. It originated at UC Berkeley's Sky Computing Lab and is now developed by a large community of contributors from academia and industry. It is written in Python and released under the Apache-2.0 license.

Speed comes from techniques such as PagedAttention for attention key-value memory, batching incoming requests continuously, chunked prefill, prefix caching, CUDA and HIP graphs, and optimized attention and mixture-of-experts kernels. It supports many quantization formats, including FP8, INT8, INT4, GPTQ and AWQ, as well as speculative decoding and disaggregated prefill and decode.

On the usability side, vLLM integrates with Hugging Face models and supports parallel sampling, beam search, streaming outputs, structured outputs and tool calling, plus several parallelism modes (tensor, pipeline, data, expert and context) for distributed inference. It targets NVIDIA, AMD and TPU hardware and suits teams that serve open models in production and need high throughput.

Key features

  • PagedAttention and continuous batching
  • Quantization including FP8, INT8, GPTQ and AWQ
  • Speculative decoding support
  • Tensor, pipeline and expert parallelism
  • OpenAI-compatible API server
  • Hugging Face model integration
  • Streaming and structured outputs

Pricing: Free and open source under the Apache-2.0 license.

Read more about vLLMWebsiteGitHub#1 Amazon SageMaker alternative

SGLang

SGLang is an open-source framework for serving language, vision-language and diffusion models, built for fast inference at scale.

GitHub stars
39k
Last commit
yesterday
Latest release
v0.5.23
Licence
Apache-2.0
Self-hosted
Yes
sglang.ioSGLang homepage screenshot

SGLang is an inference framework, released as open source, for language models, vision-language models and diffusion models. It is optimized for agentic workloads, reinforcement-learning rollouts and large-scale serving. The project includes SGLang Diffusion, a built-in engine for image and video generation that ships in the same repository and Python package. It is released under the Apache-2.0 license.

Getting started takes either a prebuilt Docker image or a Python install using uv, followed by a launch command for the chosen model. A cookbook helps pick a model and hardware pair and generates a ready-to-run command. Supported hardware spans NVIDIA and AMD GPUs, Google TPUs, Intel GPUs and CPUs, Apple Silicon through Metal and MLX, and Huawei Ascend NPUs, with more integrations in progress. The wider SGLang ecosystem adds educational projects and community events.

Key features

  • Serving for LLMs, vision-language and diffusion models
  • Docker image and uv-based Python install
  • Cookbook with ready-to-run launch commands
  • Runs on NVIDIA, AMD, TPU, Intel and Ascend hardware
  • Built-in image and video generation engine
  • Tuned for agentic and RL rollout workloads

Pricing: Free and open source under the Apache-2.0 license.

Read more about SGLangWebsiteGitHub#2 Amazon SageMaker alternative

Airflow

An Apache workflow orchestration platform where data teams author, schedule and monitor pipelines as Python code, widely used for data engineering and ELT jobs.

GitHub stars
47k
Last commit
today
Latest release
3.4.0
Licence
Apache-2.0
Self-hosted
Yes
airflow.apache.orgAirflow homepage screenshot

Apache Airflow is a workflow platform where you write, schedule and watch over pipelines in code. Workflows are defined as Python code in the form of directed acyclic graphs, or DAGs, where each task and its dependencies are explicit, so pipelines can be versioned, tested and reviewed like any other software.

A scheduler runs tasks on a defined cadence and workers execute them, while a web interface shows the status of each run, lets you inspect logs and retry failures. Airflow is used heavily in data engineering, including data integration, ELT and ETL pipelines, data orchestration and machine learning workflows, and its topics cover data science and automation. It has a large ecosystem of providers and operators for connecting to databases, cloud services and other systems, and the 3.x line is the current series.

Airflow is an Apache Software Foundation project under the Apache-2.0 license and runs on your own infrastructure, from a single machine to Kubernetes. Several vendors offer managed Airflow services, but the project itself is self-hosted software. It suits data engineers who prefer code-first orchestration over drag-and-drop tools.

Key features

  • Workflows defined as Python DAGs
  • Scheduler for recurring and dependent tasks
  • Web UI for monitoring runs and logs
  • Retries and task dependency management
  • Large provider ecosystem of integrations
  • Runs on single machines or Kubernetes

Pricing: Free and open source under the Apache-2.0 license.

Read more about AirflowWebsiteGitHub#3 Amazon SageMaker alternative

Ray

Ray is an open source framework for managing, executing, and optimizing compute needs, designed to support any AI or ML workload.

GitHub stars
44k
Last commit
yesterday
Latest release
ray-2.60.0
Licence
Apache-2.0
Self-hosted
Yes
Hosted version
Available
ray.ioRay homepage screenshot

Ray is an AI compute engine that consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. It is Python-native and built by developers for developers, designed to orchestrate infrastructure for distributed workloads on any accelerator at any scale.

The platform aims to address the AI Complexity Wall by supporting any AI or ML workload, any data types and model architectures, and heterogeneous GPUs and CPUs with fine-grained scaling. It can scale from a laptop to thousands of GPUs.

Key features

  • Supports any AI or ML workload
  • Handles multi-modal data processing including images, videos, audio
  • Runs distributed training for Gen AI foundation models, time series models, and traditional AI/ML models
  • Offers model serving with independent scaling and fractional resources via Ray Serve
  • Supports batch inference with heterogeneous compute using CPUs and GPUs
Read more about RayWebsiteGitHub#4 Amazon SageMaker alternative

MLflow

MLflow is an open-source platform for debugging, evaluating and monitoring AI agents, LLM applications and machine learning models.

GitHub stars
29k
Last commit
yesterday
Latest release
v3.18.0
Licence
Apache-2.0
Self-hosted
Yes
Hosted version
Available
mlflow.orgMLflow homepage screenshot

MLflow is an open-source AI engineering platform for agents, LLM applications and ML models. It helps teams debug, evaluate, monitor and optimize AI applications in production while keeping control of costs and access to models and data. It is written in Python, released under the Apache-2.0 license, and has been developed since 2018, with topics covering MLOps, model management and LLMOps.

For LLM and agent work it offers production observability with traces, plus evaluation, prompt management and prompt optimization, and an AI Gateway that governs costs and model access. It works with Python, TypeScript and JavaScript, Java and other languages, and integrates with OpenTelemetry and MCP. Getting started means starting an MLflow server, enabling logging in your code and running it, then exploring traces and metrics in the web UI on port 5000. A setup wizard can connect to an MLflow server or a Databricks workspace and add tracing with a coding agent.

Key features

  • Tracing and observability for LLM apps and agents
  • Evaluation tools for models and agents
  • Prompt management and prompt optimization
  • AI Gateway for cost and model access control
  • OpenTelemetry and MCP integration
  • Web UI for exploring traces and metrics

Pricing: Free and open source under the Apache-2.0 license; the README also mentions connecting to a Databricks workspace.

Read more about MLflowWebsiteGitHub#5 Amazon SageMaker alternative

Flyte

Flyte is a durable AI runtime that orchestrates training, serving, and agent workloads with automatic failure recovery.

GitHub stars
7.8k
Last commit
yesterday
Latest release
v2.0.54
Licence
Apache-2.0
Self-hosted
Yes
Hosted version
Available
flyte.orgFlyte homepage screenshot

Flyte is a distributed orchestration platform for AI workloads, including training, serving, and agents. It runs on Kubernetes and supports writing workflows in pure Python with async/await, as well as other languages like Rust, Go, and TypeScript. The platform is designed to recover from infrastructure and code failures automatically.

Workflows are defined as plain async functions without DSL or YAML. A TaskEnvironment declares images, resources, secrets, and retry policies. The runtime records I/O for every step so that a crash resumes instead of restarting. The same code can run locally, on a devbox, or on a cluster.

Key features

  • Type-checked I/O between tasks using language-native types, files, DataFrames, and dataclasses
  • Native agents that wrap tools as tasks with traced LLM calls
  • Automatic recovery from API rate limits, network errors, OOM kills, node interruptions, GPU disappears, and code exceptions
  • Supports batch inference, fine-tuning, pretraining, and RL agent loops
  • Real-time model endpoints and dashboards served with the same Python API
Read more about FlyteWebsiteGitHub#6 Amazon SageMaker alternative

BentoML

BentoML is an open-source framework for serving AI and ML models and custom inference pipelines, with a hosted platform for scaling them.

GitHub stars
8.9k
Last commit
11 days ago
Latest release
v1.4.40
Licence
Apache-2.0
Self-hosted
Yes
Hosted version
Available
bentoml.comBentoML homepage screenshot

BentoML Open-Source is presented as a flexible way to serve AI and ML models and custom inference pipelines in production. A model is packaged as a Python service using decorators, where the code declares the image, the GPU type and an API function, as in the vLLM example on the homepage.

The same company sells the Bento Inference Platform for managing, monitoring and optimizing inference, and Bento Cloud for GPU compute. Deployments can run in your own cloud, on-premises or on Kubernetes. The homepage names vLLM, TRT-LLM, JAX, SGLang, PyTorch and Transformers among the frameworks it can serve.

Key features

  • Unified packaging for models of any architecture
  • Open model catalog with quick deploys
  • Platform for managing and monitoring inference
  • Cross-region and elastic scaling, including scale to zero
  • Bring your own cloud, on-prem or Kubernetes
  • GPU hardware through Bento Cloud
  • Pre-optimized open models available on release
Read more about BentoMLWebsiteGitHub#7 Amazon SageMaker alternative

Kubeflow

Kubeflow is an open source platform for deploying ML workflows on Kubernetes.

GitHub stars
16k
Last commit
1 mo ago
Latest release
redirect
Licence
Apache-2.0
Self-hosted
Yes
kubeflow.orgKubeflow homepage screenshot

Kubeflow is the Cloud Native AI platform composed of modular, open source projects that form a Kubernetes-native stack for data and AI workloads. It is designed for AI practitioners, platform administrators, and decision-makers who need modular, scalable, and extensible tools for data, AI/ML, and HPC use-cases.

The platform's mission is to bridge the Data, AI, and Cloud Native ecosystems, enabling teams to deliver more models, agents, and AI applications into production across the AI lifecycle. It is built by an open global community and aims to be the standard for Data & AI workloads on Kubernetes.

Key features

  • Runs workloads at any scale without requiring Kubernetes expertise
  • Portable across local laptops, on-premises, and any cloud
  • Manages hyperscale training jobs and high-throughput AI agents
  • Composable tools that can be mixed and matched across the AI lifecycle
  • Includes subprojects: Spark Operator, Notebooks, Trainer, Katib, and Hub
Read more about KubeflowWebsiteGitHub#8 Amazon SageMaker alternative

Pipelines

Kubeflow Pipelines, a platform for building and running portable machine learning workflows as containers on Kubernetes, authored in Python.

GitHub stars
4.3k
Last commit
yesterday
Latest release
2.18.0
Licence
Apache-2.0
Self-hosted
Yes
kubeflow.orgPipelines homepage screenshot

Kubeflow Pipelines (KFP) is a platform for building and deploying portable and scalable machine learning workflows using containers on Kubernetes-based systems. Pipelines are reusable end-to-end workflows built with the Kubeflow Pipelines SDK. Its stated goals are end-to-end orchestration, easy experimentation across many trials, and reuse of components and pipelines.

Authors write components and pipelines in Python with the KFP SDK, compile them to an intermediate representation YAML file, and submit that file to a KFP-conformant backend, such as the open source KFP backend or Google Cloud Vertex AI Pipelines. A pipeline is a directed acyclic graph of components, and each component execution corresponds to one container execution.

The open source backend is available as part of the Kubeflow Platform or as a standalone service. Pipeline execution uses the Argo Emissary executor. The README's compatibility notice says the KFP 3.0 branch supports Argo Workflows 4.0.x and 4.1.x and MySQL 8, and not Argo 3.x. The code is Go with an Apache 2.0 licence.

Key features

  • Author ML workflows natively in Python
  • Compiles pipelines to platform-neutral YAML
  • Parameters and artifacts pass between components
  • Parallel task execution and caching
  • Tracks and visualizes pipelines, runs and experiments
  • Can run standalone or within Kubeflow
  • Pipelines can run on a conformant backend such as Vertex AI
Read more about PipelinesWebsiteGitHub#9 Amazon SageMaker alternative

Polyaxon

Polyaxon is an open-source AI control plane for Kubernetes that schedules, runs, debugs, observes, and automates AI/ML workloads, agents, and sandboxes on your own infrastructure.

GitHub stars
3.7k
Last commit
yesterday
Licence
Apache-2.0
Self-hosted
Yes
polyaxon.comPolyaxon homepage screenshot

Polyaxon is an open-source AI control plane designed to run on Kubernetes. It enables scheduling, tracking, and governance of AI/ML workloads, including GPU training, agent sessions, and sandboxes, all within self-hosted deployments. The platform supports training, services, and sandboxes using shared specs, queues, governance, and data connections.

It provides a unified interface for training and fine-tuning models, serving inference, running notebooks, and managing interactive sessions. Polyaxon integrates with various frameworks, clients, and APIs, and offers governance features such as organizations, teams, authentication, audit logs, and policies. It is suitable for teams that need to manage AI workloads on their own clusters and storage backends.

Key features

  • Scheduling and orchestration of runs, experiments, and batch jobs
  • Training and fine-tuning with support for LoRA and SFT
  • Services for serving, inference, notebooks, TensorBoard, and ML apps
  • Sandboxes with SSH, tmux, interactive, and background modes
  • Governance with organizations, teams, authn/authz, audit logs, and policies
  • Execution controls via agents, queues, connections, and presets
Read more about PolyaxonWebsiteGitHub#10 Amazon SageMaker alternative

6 more Amazon SageMaker alternatives

Commercial Amazon SageMaker alternatives

Hosted, closed-source products that compete with Amazon SageMaker. Showing 12 of 25.

See every Amazon SageMaker alternative on its tool page

Amazon SageMaker alternatives: questions

What is the best open-source alternative to Amazon SageMaker?
vLLM is the top-ranked open-source alternative to Amazon SageMaker on Enlisted: A Python library and server for fast, memory-efficient LLM inference and serving, with PagedAttention, continuous batching and broad quantization support. Other strong options are SGLang, Airflow, Ray and MLflow.
What are the best commercial alternatives to Amazon SageMaker?
Hugging Face, Lightning AI, Azure AI Foundry, Baseten and Domino Data Lab are among the hosted, closed-source alternatives to Amazon SageMaker listed on Enlisted, which lists 25 in total. Compare their pricing and features on each tool page.
Are these Amazon SageMaker alternatives free?
All 16 are open source, so the code is free to use under its licence, and all of them can be self-hosted on your own server or computer. 5 also offer a paid or managed cloud version if you'd rather not host it yourself.
How is this list of Amazon SageMaker alternatives ranked?
By a score built from GitHub stars, star growth over the last 30 days and how recently the code changed. 12 of these projects shipped code in the last 30 days. Data is refreshed daily, and nobody can pay to move up.

People also look for alternatives to…

View all