#1
An open-source, event-driven orchestration and scheduling platform where data, AI and infrastructure workflows are defined declaratively in YAML and managed from a UI.
- GitHub stars
- 30k
- Last commit
- today
- Latest release
- v2.0.10
- Licence
- Apache-2.0
- Self-hosted
- Yes
go.kestra.io
Kestra is an orchestration platform, open source, for data pipelines, AI workflows and infrastructure automation. It brings scheduled and event-triggered automation together under one declarative interface that does not depend on a programming language, applying infrastructure-as-code practices to pipelines so that reliable workflows can be defined in a few lines of YAML.
Workflows can be built in the UI, written by a built-in AI Copilot, or generated from coding agents such as Claude Code and Cursor using agent skills, and everything can be kept as code with Git integration even when it was authored visually. A large plugin ecosystem connects it to external systems, and key concepts cover tasks, triggers and flows. The project is written in Java, with topics covering orchestration, high availability, pipeline-as-code and DevOps, and the 2.0 release adds new capabilities.
Kestra is Apache-2.0 licensed and can be self-hosted, with a quick start that gets a first workflow running in minutes. It targets data engineers, platform teams and automation developers who want an alternative to schedulers such as Airflow with a more declarative style.
Key features
- Declarative YAML workflow definitions
- Scheduled and event-driven triggers
- Visual editor and AI Copilot
- Git version control integration
- Large plugin ecosystem
- Agent skills for coding agents
Pricing: Free and open source under the Apache-2.0 license.
#2
An Apache workflow orchestration platform where data teams author, schedule and monitor pipelines as Python code, widely used for data engineering and ELT jobs.
- GitHub stars
- 47k
- Last commit
- today
- Latest release
- 3.4.0
- Licence
- Apache-2.0
- Self-hosted
- Yes
airflow.apache.org
Apache Airflow is a workflow platform where you write, schedule and watch over pipelines in code. Workflows are defined as Python code in the form of directed acyclic graphs, or DAGs, where each task and its dependencies are explicit, so pipelines can be versioned, tested and reviewed like any other software.
A scheduler runs tasks on a defined cadence and workers execute them, while a web interface shows the status of each run, lets you inspect logs and retry failures. Airflow is used heavily in data engineering, including data integration, ELT and ETL pipelines, data orchestration and machine learning workflows, and its topics cover data science and automation. It has a large ecosystem of providers and operators for connecting to databases, cloud services and other systems, and the 3.x line is the current series.
Airflow is an Apache Software Foundation project under the Apache-2.0 license and runs on your own infrastructure, from a single machine to Kubernetes. Several vendors offer managed Airflow services, but the project itself is self-hosted software. It suits data engineers who prefer code-first orchestration over drag-and-drop tools.
Key features
- Workflows defined as Python DAGs
- Scheduler for recurring and dependent tasks
- Web UI for monitoring runs and logs
- Retries and task dependency management
- Large provider ecosystem of integrations
- Runs on single machines or Kubernetes
Pricing: Free and open source under the Apache-2.0 license.
#3
An analytics engine for processing large datasets, with APIs in Scala, Java, Python and R and libraries for SQL, machine learning, graphs and streaming.
- GitHub stars
- 44k
- Last commit
- yesterday
- Licence
- Apache-2.0
- Self-hosted
- Yes
spark.apache.org
Apache Spark is an analytics engine that brings batch, SQL, machine learning, graph and streaming workloads together for processing data at scale. It is maintained by the Apache Software Foundation under the Apache-2.0 license. It is written mainly in Scala and offers high-level APIs in Scala, Java and Python, with an R API that is now deprecated, on top of an engine optimized to execute general computation graphs.
A set of higher-level tools is included: Spark SQL for queries and DataFrames, MLlib for machine learning, GraphX for graphs, Structured Streaming for streams, and a pandas-style API for existing pandas workloads. Together they let teams run batch analytics, machine learning and streaming jobs on a single engine and share code between them.
The README points to the project website for full documentation and programming guides, and it contains only basic setup and build instructions. Spark suits data engineers and data scientists who process datasets too large for a single machine, and organizations that want one framework for ETL, analytics and machine learning on clusters.
Key features
- High-level APIs in Scala, Java and Python
- Optimized engine for computation graphs
- Spark SQL and DataFrames
- MLlib for machine learning
- GraphX for graph analytics
- Structured Streaming for stream processing
- Pandas API on Spark
Pricing: Free and open source under the Apache-2.0 license.
#4
A Python workflow orchestration framework for turning scripts into resilient, observable data pipelines with scheduling, retries, caching and event-driven automation.
- GitHub stars
- 24k
- Last commit
- today
- Latest release
- 3.9.2
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
prefect.io
Prefect is a Python framework for orchestrating workflows and building data pipelines. It aims to be the simplest way to lift an ordinary script into a production workflow: you add flow and task decorators, and Prefect adds the machinery that makes pipelines dependable and visible.
That machinery includes scheduling, caching, retries and event-based automations, and flows can express dependencies and complex branching logic. The goal is dynamic pipelines that react to changes in the world and recover from unexpected failures. Run activity is tracked, and you can monitor it from a self-hosted Prefect server or from the managed Prefect Cloud dashboard. Prefect requires Python 3.10 or newer, and the documentation covers installation, quickstart, building and deploying workflows and agent setup.
Prefect is licensed under Apache-2.0. Topics list data engineering, ML ops and observability among its uses. It is aimed at data teams who prefer to write workflows as plain Python code, and who want a choice between running the orchestrator themselves or using a hosted control plane.
Key features
- Flows and tasks defined with Python decorators
- Scheduling and event-based automations
- Automatic retries and caching
- Complex branching and dependencies
- Self-hosted server or Prefect Cloud monitoring
- Deployment of workflows to infrastructure
Pricing: Prefect Cloud's Hobby tier is free. Starter is $100 per month and Team $100 per user per month, both billed monthly; Enterprise is custom and billed annually.
#5
An open-source data movement platform with hundreds of connectors for ELT pipelines from APIs, databases and files into warehouses, lakes and AI applications.
- GitHub stars
- 22k
- Last commit
- today
- Latest release
- v2.0.0
- Self-hosted
- Yes
- Hosted version
- Available
airbyte.com
Airbyte is an open-source data movement platform for moving data from APIs, databases and files into data warehouses, data lakes and AI applications. Its premise is that only an open-source project can reach the long tail of data sources while letting engineers tailor existing connectors, with the aim of moving data from any source to any destination.
It offers a catalog of 600 or more connectors for APIs, databases, warehouses, lakes and AI applications, with warehouse destinations such as BigQuery, Redshift and Snowflake, and topics also mention change data capture. The README distinguishes products by job: Airbyte Open Source or Airbyte Cloud for ELT and ETL into warehouses, lakes or databases, and Airbyte Agents, a managed data and context layer, plus an open-source Agent SDK for giving AI agents and MCP clients real-time access to business data.
Airbyte is written in Python and Java, and its license is listed as 'Other' on GitHub because the project uses a mix of licenses, so review the terms for your use. It can be self-hosted or run as the managed Airbyte Cloud, and suits data teams that need many integrations without writing each pipeline by hand.
Key features
- 600+ source and destination connectors
- ELT pipelines into warehouses and lakes
- Change data capture support
- Customizable open-source connectors
- Agent SDK for AI agent data access
- Self-hosted or Airbyte Cloud
Pricing: The open-source Core edition is free to self-host. Managed cloud starts at $20 a month (Standard) or $189 a month for 40 credits (Plus); Pro and Enterprise Flex are custom. 30-day trial.
#6
A Python data orchestration platform for building, scheduling and observing data pipelines around the data assets they produce; Dagster is now part of Prefect.
- GitHub stars
- 16k
- Last commit
- today
- Latest release
- 1.13.30
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
dagster.io
Dagster is an orchestration platform for developing, running and observing data assets. Instead of thinking only in terms of tasks, teams define the tables, files and models their pipelines produce, and Dagster tracks how they are built, scheduled and monitored. It is written in Python and aimed at data engineering, analytics and machine learning workloads.
The product surface covers data orchestration, a data catalog, data quality checks, cost insights and many integrations, with an enterprise offering for larger organizations. Comparison pages on the website position it against Airflow, dbt Cloud, Azure Data Factory and AWS Step Functions, and topics include ETL, scheduling and MLOps. The homepage now announces that Dagster is part of Prefect, and points visitors looking for agentic orchestration or MCP support to Prefect.
The open-source core is Apache-2.0 licensed and can be self-hosted, while a managed cloud and enterprise plans are offered with pricing on the company site. Dagster University and documentation help with learning. It suits data teams that want a software-engineering-style approach to pipelines.
Key features
- Asset-centric data orchestration
- Scheduling and monitoring of pipelines
- Data catalog and data quality features
- Cost insights for pipelines
- Integrations with common data tools
- Pipelines developed in Python
Pricing: Dagster+ Solo is $10 and Starter $100 per month, each plus per-credit usage charges, with a 30-day free trial. Pro is quoted through sales.
#7
Argo Workflows is an open-source, container-native workflow engine for orchestrating parallel jobs on Kubernetes, implemented as a CRD.
- GitHub stars
- 17k
- Last commit
- yesterday
- Latest release
- v4.1.6
- Licence
- Apache-2.0
- Self-hosted
- Yes
argo-workflows.readthedocs.io
Argo Workflows is a container-native workflow engine for running parallel jobs on Kubernetes. It is implemented as a Kubernetes Custom Resource Definition, so workflows are defined as Kubernetes resources in which each step is a container.
Multi-step workflows can be modeled as a sequence of tasks or as a directed acyclic graph that captures dependencies between tasks. Typical uses include machine learning pipelines, data and batch processing, infrastructure automation and CI/CD. Client libraries exist for Java, Go, Python through the Hera SDK, and TypeScript.
The project is a Cloud Native Computing Foundation graduated project written in Go and licensed under Apache 2.0. It is cloud agnostic and runs on any Kubernetes cluster, and the ecosystem includes tools such as Argo Events, Kubeflow Pipelines and Katib that use or build on it. Interactive training material and a demo environment are linked for trying it out.
Key features
- Workflows defined as Kubernetes custom resources
- Each step runs as a container
- DAG and sequential task modeling
- Parallel job execution on Kubernetes
- Client libraries for Java, Go, Python and TypeScript
- Runs on any Kubernetes cluster
Pricing: Free and open source under the Apache 2.0 license.
#8
Apache Beam is an open-source programming model for batch and streaming data processing whose pipelines run on multiple execution engines.
- GitHub stars
- 8.7k
- Last commit
- yesterday
- Latest release
- v2.77.0
- Licence
- Apache-2.0
beam.apache.org
Apache Beam provides a single programming model for both batch and streaming data processing. Pipelines read from a range of supported sources, apply business logic, and write results to common data sinks. Quickstarts are offered for Java, Python and Go.
Beam pipelines can execute on multiple execution environments, called runners, which the project presents as a way to avoid lock-in. It is extensible, with projects such as TensorFlow Extended and Apache Hop built on top of it, and it supports multi-language pipelines. The Beam Playground lets people try transforms and examples without installing anything locally.
The project publishes case studies from several organisations. LinkedIn is described as running streaming infrastructure on Beam, OCTO used it to move a French grocery retailer to streaming processing of transactional data, HSBC used it as a computational platform and risk engine, and Booking.com uses it for global ad bidding. The site also invites other users to share their own story.
Key features
- One programming model for batch and streaming
- Pipelines run on several runners
- Reads from many sources on-prem or in the cloud
- Extended by projects such as TensorFlow Extended
- Interactive Beam Playground
#9
Meltano is an open source data integration platform that extracts and loads data into a warehouse using pipelines defined as code.
- GitHub stars
- 2.7k
- Last commit
- yesterday
- Latest release
- v4.4.1
- Licence
- MIT
- Self-hosted
- Yes
- Hosted version
- Available
meltano.com
Meltano is an open source platform for moving data from sources into a warehouse and onwards to analytics and AI. Pipelines are defined as code in a project, and release management and governance follow Git and GitHub workflows.
It offers pre-built connectors for SaaS apps, APIs, files and databases, 600+ sources and destinations in total, usable from a UI, CLI, API or AI. Transformation uses dbt and Elementary, configured and version-controlled within the same Meltano project, and the output can feed BI tools, semantic layers and notebooks.
Developers can build their own Singer connectors with the Meltano SDK and extensions with the EDK. You can self-manage Meltano or use a managed orchestrator; orchestration with schedules, triggers and retries is built in on Meltano Cloud. The site says it is free to use with paid support options.
Key features
- 600+ connectors for sources and destinations
- dbt and Elementary models kept in the project
- Git-based release management and governance
- Scheduling, triggers and retries on Meltano Cloud
- SDK for building Singer connectors
- Output feeds BI tools, semantic layers and notebooks
Pricing: Open source and free to use, with paid support options.
#10
Singer is an open-source standard for scripts that move data, where extraction scripts called taps pipe JSON to loading scripts called targets.
- GitHub stars
- 1.4k
- Last commit
- 1 yr ago
singer.io
Singer defines how data extraction scripts, called taps, and data loading scripts, called targets, talk to each other. Because the format is shared, any tap can be paired with any target to move data from a source to a destination.
Taps and targets are small applications chained with Unix pipes, so no daemon or complicated plugin system is needed. They exchange JSON, which makes them easy to write in any programming language, and JSON Schema can describe richer types when a strict structure is needed.
The site shows two commands that pull exchange rate data into a CSV file. Its taps list covers many sources, including PostgreSQL, MySQL, Salesforce, Stripe, HubSpot and Google Analytics, and its targets include CSV, Google BigQuery and Google Sheets. A page explains how to develop a new tap.
Key features
- Taps pull data from a source into a JSON stream
- Targets load data into files, APIs or databases
- JSON messages usable from any programming language
- State kept between runs for incremental extraction
- Two-command example exports exchange rates to CSV
- Moves data between databases, web APIs, files and queues
Commercial Astronomer alternatives
Hosted, closed-source products that compete with Astronomer.
Astronomer alternatives: questions
- What is the best open-source alternative to Astronomer?
- Kestra is the top-ranked open-source alternative to Astronomer on Enlisted: An open-source, event-driven orchestration and scheduling platform where data, AI and infrastructure workflows are defined declaratively in YAML and managed from a UI. Other strong options are Airflow, Apache Spark, Prefect and Airbyte.
- What are the best commercial alternatives to Astronomer?
- AWS Glue, Azure Data Factory, Matillion, Talend and Google Cloud Dataflow are among the hosted, closed-source alternatives to Astronomer listed on Enlisted, which lists 12 in total. Compare their pricing and features on each tool page.
- Are these Astronomer alternatives free?
- All 10 are open source, so the code is free to use under its licence, and all of them can be self-hosted on your own server or computer. 4 also offer a paid or managed cloud version if you'd rather not host it yourself.
- How is this list of Astronomer alternatives ranked?
- By a score built from GitHub stars, star growth over the last 30 days and how recently the code changed. 9 of these projects shipped code in the last 30 days. Data is refreshed daily, and nobody can pay to move up.