Pipelines

Open source

Kubeflow Pipelines, a platform for building and running portable machine learning workflows as containers on Kubernetes, authored in Python.

Open-source alternative to

kubeflow.org
Pipelines homepage screenshot
GitHub stars
4.3k
Last 30 days
+24(+1%)
Last commit
yesterday
Repository age
8 years
Version
2.18.0
Licence
Apache-2.0
Self-hosted
Yes

About Pipelines

Kubeflow Pipelines (KFP) is a platform for building and deploying portable and scalable machine learning workflows using containers on Kubernetes-based systems. Pipelines are reusable end-to-end workflows built with the Kubeflow Pipelines SDK. Its stated goals are end-to-end orchestration, easy experimentation across many trials, and reuse of components and pipelines.

Authors write components and pipelines in Python with the KFP SDK, compile them to an intermediate representation YAML file, and submit that file to a KFP-conformant backend, such as the open source KFP backend or Google Cloud Vertex AI Pipelines. A pipeline is a directed acyclic graph of components, and each component execution corresponds to one container execution.

The open source backend is available as part of the Kubeflow Platform or as a standalone service. Pipeline execution uses the Argo Emissary executor. The README's compatibility notice says the KFP 3.0 branch supports Argo Workflows 4.0.x and 4.1.x and MySQL 8, and not Argo 3.x. The code is Go with an Apache 2.0 licence.

Key features

  • Author ML workflows natively in Python
  • Compiles pipelines to platform-neutral YAML
  • Parameters and artifacts pass between components
  • Parallel task execution and caching
  • Tracks and visualizes pipelines, runs and experiments
  • Can run standalone or within Kubeflow
  • Pipelines can run on a conformant backend such as Vertex AI

Good fit for

  • End-to-end ML workflow orchestration
  • Experiment tracking across many trials
  • Reusing ML components between pipelines
  • Running pipelines locally during development
Tags
machine-learning
mlops
kubeflow
kubernetes
pipelines
workflow-orchestration
python

Pipelines: questions and answers

What is Pipelines used for?
Pipelines is a platform for building and running portable machine learning workflows as containers on Kubernetes, authored in Python. It is a good fit for end-to-end ML workflow orchestration, experiment tracking across many trials and reusing ML components between pipelines.
Is Pipelines open source?
Yes. Pipelines is open source under the Apache-2.0 licence. Its source code is on GitHub at kubeflow/pipelines and is written mainly in Python.
Is Pipelines free?
Yes. Pipelines is open source, so the software itself is free to use.
Can I self-host Pipelines?
Yes. Pipelines can be self-hosted on your own server or infrastructure.
What is Pipelines an alternative to?
Pipelines is an open-source alternative to Amazon SageMaker. Other open-source alternatives to Amazon SageMaker include KubeDL, Kubeflow and Polyaxon.
Is Pipelines actively maintained?
Yes. The most recent commit to Pipelines was on 12 November 2026, and the latest release is 2.18.0, published on 28 October 2026. The project has 4.3k stars on GitHub, with 24 added in the last 30 days.

Open-source alternatives to Pipelines

See all

SaaS alternatives to Pipelines

See all