#1
An Apache workflow orchestration platform where data teams author, schedule and monitor pipelines as Python code, widely used for data engineering and ELT jobs.
- GitHub stars
- 47k
- Last commit
- today
- Latest release
- 3.4.0
- Licence
- Apache-2.0
- Self-hosted
- Yes
airflow.apache.org
Apache Airflow is a workflow platform where you write, schedule and watch over pipelines in code. Workflows are defined as Python code in the form of directed acyclic graphs, or DAGs, where each task and its dependencies are explicit, so pipelines can be versioned, tested and reviewed like any other software.
A scheduler runs tasks on a defined cadence and workers execute them, while a web interface shows the status of each run, lets you inspect logs and retry failures. Airflow is used heavily in data engineering, including data integration, ELT and ETL pipelines, data orchestration and machine learning workflows, and its topics cover data science and automation. It has a large ecosystem of providers and operators for connecting to databases, cloud services and other systems, and the 3.x line is the current series.
Airflow is an Apache Software Foundation project under the Apache-2.0 license and runs on your own infrastructure, from a single machine to Kubernetes. Several vendors offer managed Airflow services, but the project itself is self-hosted software. It suits data engineers who prefer code-first orchestration over drag-and-drop tools.
Key features
- Workflows defined as Python DAGs
- Scheduler for recurring and dependent tasks
- Web UI for monitoring runs and logs
- Retries and task dependency management
- Large provider ecosystem of integrations
- Runs on single machines or Kubernetes
Pricing: Free and open source under the Apache-2.0 license.
#2
An analytics engine for processing large datasets, with APIs in Scala, Java, Python and R and libraries for SQL, machine learning, graphs and streaming.
- GitHub stars
- 44k
- Last commit
- yesterday
- Licence
- Apache-2.0
- Self-hosted
- Yes
spark.apache.org
Apache Spark is an analytics engine that brings batch, SQL, machine learning, graph and streaming workloads together for processing data at scale. It is maintained by the Apache Software Foundation under the Apache-2.0 license. It is written mainly in Scala and offers high-level APIs in Scala, Java and Python, with an R API that is now deprecated, on top of an engine optimized to execute general computation graphs.
A set of higher-level tools is included: Spark SQL for queries and DataFrames, MLlib for machine learning, GraphX for graphs, Structured Streaming for streams, and a pandas-style API for existing pandas workloads. Together they let teams run batch analytics, machine learning and streaming jobs on a single engine and share code between them.
The README points to the project website for full documentation and programming guides, and it contains only basic setup and build instructions. Spark suits data engineers and data scientists who process datasets too large for a single machine, and organizations that want one framework for ETL, analytics and machine learning on clusters.
Key features
- High-level APIs in Scala, Java and Python
- Optimized engine for computation graphs
- Spark SQL and DataFrames
- MLlib for machine learning
- GraphX for graph analytics
- Structured Streaming for stream processing
- Pandas API on Spark
Pricing: Free and open source under the Apache-2.0 license.
#3
Apache Kafka is an open-source platform for distributed event streaming, used in data pipelines, streaming analytics and data integration.
- GitHub stars
- 34k
- Last commit
- yesterday
- Licence
- Apache-2.0
- Self-hosted
- Yes
kafka.apache.org
Apache Kafka is a platform for distributed event streaming. Teams use it to move and process streams of events for data pipelines, streaming analytics, data integration and business-critical applications. The code lives in the Apache Software Foundation repository, is written mainly in Java with Scala for parts of the broker, and is released under the Apache-2.0 license.
Kafka is installed software that you run yourself. The project builds and tests against recent Java versions, supports Scala 2.13 only, and documents how to build a JAR and start a broker through its quickstart. The repository also contains client and streams modules, along with test tooling for unit and integration runs. Because Kafka is operated as infrastructure, adopters typically run clusters on their own servers or on Kubernetes, though managed services from other vendors exist.
Key features
- Distributed event streaming platform
- Durable publish and subscribe topics
- Client libraries and a streams module
- Suited to data pipelines and integration
- Runs on Java with Scala 2.13
- Quickstart for building and running a broker
Pricing: Free and open source under the Apache-2.0 license.
#4
An open-source data movement platform with hundreds of connectors for ELT pipelines from APIs, databases and files into warehouses, lakes and AI applications.
- GitHub stars
- 22k
- Last commit
- today
- Latest release
- v2.0.0
- Self-hosted
- Yes
- Hosted version
- Available
airbyte.com
Airbyte is an open-source data movement platform for moving data from APIs, databases and files into data warehouses, data lakes and AI applications. Its premise is that only an open-source project can reach the long tail of data sources while letting engineers tailor existing connectors, with the aim of moving data from any source to any destination.
It offers a catalog of 600 or more connectors for APIs, databases, warehouses, lakes and AI applications, with warehouse destinations such as BigQuery, Redshift and Snowflake, and topics also mention change data capture. The README distinguishes products by job: Airbyte Open Source or Airbyte Cloud for ELT and ETL into warehouses, lakes or databases, and Airbyte Agents, a managed data and context layer, plus an open-source Agent SDK for giving AI agents and MCP clients real-time access to business data.
Airbyte is written in Python and Java, and its license is listed as 'Other' on GitHub because the project uses a mix of licenses, so review the terms for your use. It can be self-hosted or run as the managed Airbyte Cloud, and suits data teams that need many integrations without writing each pipeline by hand.
Key features
- 600+ source and destination connectors
- ELT pipelines into warehouses and lakes
- Change data capture support
- Customizable open-source connectors
- Agent SDK for AI agent data access
- Self-hosted or Airbyte Cloud
Pricing: The open-source Core edition is free to self-host. Managed cloud starts at $20 a month (Standard) or $189 a month for 40 credits (Plus); Pro and Enterprise Flex are custom. 30-day trial.
#5
Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams, designed to run in all common cluster environments.
- GitHub stars
- 26k
- Last commit
- yesterday
- Licence
- Apache-2.0
- Self-hosted
- Yes
flink.apache.org
Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. It has been designed to run in all common cluster environments, perform computations at in-memory speed and at any scale.
Flink provides correctness guarantees including exactly-once state consistency, event-time processing, and sophisticated late data handling. It offers layered APIs such as SQL on stream and batch data, DataStream API, and ProcessFunction for time and state.
Key features
- Exactly-once state consistency
- Event-time processing
- Sophisticated late data handling
- Layered APIs including SQL, DataStream API, and ProcessFunction
- Flexible deployment and high-availability setup
- Scale-out architecture with support for very large state and incremental checkpoints
#6
A data transformation framework that lets analysts and engineers build analytics models with software-engineering practices; v2.0 is a ground-up rewrite in Rust.
- GitHub stars
- 14k
- Last commit
- today
- Latest release
- v1.12.7
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
getdbt.com
dbt is a framework for transforming data in a warehouse using the same practices that software engineers apply to applications. Analysts and analytics engineers write models, and dbt parses, compiles and runs the project so that transformations are version-controlled and repeatable.
The repository now holds the Apache 2.0 source of dbt v2.0, a ground-up rewrite in Rust. According to its README, v2.0 parses and compiles projects much faster than v1, enforces a stricter language specification at parse time, produces Parquet artifacts alongside the existing JSON ones, and ships as one standalone executable, so no Python installation is needed. The local documentation experience was rebuilt around those artifacts to handle large projects.
The Apache 2.0 code is available to everyone, while the packaged dbt distribution adds dbt-specific customizations under a separate dbt product license. v2.0 supports macOS and Linux on x86-64 and ARM. The v1 Python implementation continues on the 1.latest branch. dbt Labs also offers dbt as a commercial cloud platform.
Key features
- Data transformation using software engineering practices
- Faster parsing and compiling in v2.0
- Strict language specification checks at parse time
- Parquet artifacts for querying project metadata
- Single binary without a Python runtime
- Rebuilt local documentation experience
Pricing: A single-seat Developer plan to start free. Starter costs $100 per user per month; Enterprise and Enterprise+ are custom. A trial is available and add-ons are billed on usage.
#7
OpenRefine is a free, open source tool for cleaning messy data, transforming formats, and extending data with web services and external sources.
- GitHub stars
- 12k
- Last commit
- yesterday
- Latest release
- 3.10.1
- Licence
- BSD-3-Clause
- Self-hosted
- Yes
openrefine.org
OpenRefine is a free, open source desktop application designed for working with messy data. It allows users to clean data, transform it from one format to another, and extend it using web services and external data sources.
The tool is aimed at anyone who needs to prepare and improve datasets, from journalists to researchers. It emphasizes privacy by processing data locally on the user's machine.
Key features
- Faceting: drill through large datasets using facets and apply operations on filtered views.
- Clustering: fix inconsistencies by merging similar values using heuristics.
- Reconciliation: match datasets to external databases via reconciliation services.
- Infinite undo/redo: rewind to any previous state and replay operation history on a new version.
- Privacy: data is cleaned on the user's machine, not in a cloud service.
- Wikibase: contribute to Wikidata and other Wikibase instances.
#8
An open-source ETL and ELT platform built on DuckDB that you deploy yourself, with visual or SQL pipelines, dbt, CDC, data quality and an MCP server.
- GitHub stars
- 1.6k
- Last commit
- today
- Latest release
- v0.8.0
- Licence
- Apache-2.0
- Self-hosted
- Yes
duckle.org
Duckle is an open-source ETL and ELT platform for teams that want their data pipelines running on their own infrastructure. You build pipelines on a visual canvas, in Python or in SQL, then ship the same file to your own server or cloud account, where a headless runner executes it on a schedule.
Pipelines compile to SQL on DuckDB and use all the cores available, so a larger machine runs them faster. The platform lists support for a large catalog of components, dbt, change data capture, data quality checks, reverse ETL and lineage, along with a web console, roles, an audit trail and an MCP server so AI agents such as Claude or Cursor can work with it. Each pipeline is a single file that can live in git.
Duckle is written in Rust and licensed under Apache-2.0, and it runs headless in Docker or on a plain server, with Kubernetes among its topics. It states there is no vendor cloud and no per-row billing. It is an independent project by SlothFlowLabs and is not affiliated with DuckDB Labs or MotherDuck. It suits data engineers who want a self-hosted alternative to managed pipeline services.
Key features
- Visual canvas, Python, or SQL pipeline authoring
- Runs on DuckDB across all CPU cores
- dbt, CDC, and reverse ETL support
- Data quality checks and lineage
- Web console with roles and audit trail
- MCP server for AI agents
Pricing: Free and open source under the Apache-2.0 licence, with no vendor cloud or per-row billing according to the README.
#9
Mage is an open-source data pipeline tool for building, scheduling and debugging ETL and transformation jobs in Python, SQL or R through a notebook-style UI.
- GitHub stars
- 8.8k
- Last commit
- 20 days ago
- Latest release
- 0.9.80
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
mage.ai
Mage, called Mage OSS in its README, is a self-hosted development environment for building data pipelines. It is meant for teams that automate ETL tasks, design data flows or orchestrate transformations, and it presents the work in a notebook-style interface made of modular blocks of code.
Pipelines are written block by block in Python, SQL or R. Jobs can be run manually or on a schedule, including cron expressions, and prebuilt connectors reach databases, APIs and cloud storage. Debugging is visual, with logs, live data previews and execution you can follow step by step, and dbt models can be built and run inside Mage. Topics on the repository also reference Spark, reverse ETL and machine learning workloads.
Mage OSS is licensed under Apache-2.0 and installs with Docker, pip or conda, with no cloud account required. For larger deployments the vendor offers Mage Pro, a commercial platform that adds enterprise orchestration, collaboration and AI-assisted workflows.
Key features
- Modular pipelines in Python, SQL or R
- Notebook-style interactive editor
- Prebuilt connectors for databases, APIs and storage
- Manual and cron-based scheduling
- Visual debugging with logs and previews
- dbt model support inside pipelines
Pricing: Managed Mage starts at $29 per month (Starter) and $100 per month (Team), plus $0.50 per CPU-hour, with a 7-day free trial. Enterprise is by contract.
#10
An open-source data pipeline tool that combines ingestion, SQL and Python transformations, and quality checks in one framework you can run locally or in CI.
- GitHub stars
- 1.8k
- Last commit
- today
- Latest release
- v0.11.821
- Licence
- Apache-2.0
- Self-hosted
- Yes
getbruin.com
Bruin is a data pipeline tool that brings data ingestion, transformation and data quality into a single framework. Instead of stitching together separate tools for loading, modeling and testing, you describe a pipeline with SQL, Python or R assets and run it against the major data platforms, such as BigQuery and Snowflake.
Pipelines can ingest data from different sources, materialize tables and views, and build incremental tables. Python assets run in isolated environments using uv, Jinja templating cuts down on repetition, and built-in quality checks validate results. A dry-run mode validates a pipeline end to end, secrets are injected through environment variables, and a VS Code extension supports day-to-day development.
Bruin is written in Go, licensed under Apache-2.0 and distributed as a command-line tool that is easy to install. It runs on your local machine, an EC2 instance or GitHub Actions, so no dedicated orchestration server is required. It suits data engineers and analytics teams who want pipelines defined as code in a repository.
Key features
- SQL, Python, and R transformations
- Data ingestion from multiple sources
- Table and view materializations with incremental loads
- Built-in data quality checks
- Dry-run validation of whole pipelines
- VS Code extension for development
Pricing: Free and open source under the Apache-2.0 licence.
1 more Talend alternative
Talend alternatives: questions
- What is the best open-source alternative to Talend?
- Airflow is the top-ranked open-source alternative to Talend on Enlisted: An Apache workflow orchestration platform where data teams author, schedule and monitor pipelines as Python code, widely used for data engineering and ELT jobs. Other strong options are Apache Spark, Kafka, Airbyte and Flink.
- What are the best commercial alternatives to Talend?
- AWS Glue, IBM DataStage, Azure Data Factory, Matillion and Pentaho are among the hosted, closed-source alternatives to Talend listed on Enlisted, which lists 27 in total. Compare their pricing and features on each tool page.
- Are these Talend alternatives free?
- All 11 are open source, so the code is free to use under its licence, and all of them can be self-hosted on your own server or computer. 3 also offer a paid or managed cloud version if you'd rather not host it yourself.
- How is this list of Talend alternatives ranked?
- By a score built from GitHub stars, star growth over the last 30 days and how recently the code changed. 11 of these projects shipped code in the last 30 days. Data is refreshed daily, and nobody can pay to move up.