Open-source data pipeline & ETL tools. To compare with crawlable.app.

Airflow

Data Pipelines & ETL

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Apache Spark

Data Pipelines & ETL

Apache Spark - A unified analytics engine for large-scale data processing

Kafka

Data Pipelines & ETL

Apache Kafka - A distributed event streaming platform

Kestra

Data Pipelines & ETL

Event Driven Orchestration & Scheduling Platform for Mission Critical Applications

Prefect

Data Pipelines & ETL

Prefect is a workflow orchestration framework for building resilient data pipelines in Pyt

Airbyte

Data Pipelines & ETL

Airbyte is the open-source data movement platform. Run ELT pipelines across 700+ connector

Dagster

Data Pipelines & ETL

An orchestration platform for the development, production, and observation of data assets.

Logstash

Data Pipelines & ETL

Logstash - transport and process your logs, events, or other data

dbt

Data Pipelines & ETL

dbt enables data analysts and engineers to transform their data using the same practices t

Debezium

Data Pipelines & ETL

Change data capture for a variety of databases. Please log issues at https://github.com/de

RisingWave

Data Pipelines & ETL

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event s

Delta Lake

Data Pipelines & ETL

An open-source storage framework that enables building a Lakehouse architecture with compu

Mage AI

Data Pipelines & ETL

🧙 Build, run, and manage data pipelines for integrating and transforming data.

Apache Zeppelin

BI & Dashboards

Web-based notebook that enables data-driven, interactive data analytics and collaborative

Materialize

Databases

The live data layer for apps and AI agents. Create up-to-the-second views into your busine

Apache Hive

Databases

Apache Hive

CKAN

Data Pipelines & ETL

CKAN is an open-source DMS (data management system) for powering data hubs and data portal

Jitsu

Data Pipelines & ETL

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for mo

esProc SPL is a JVM-based programming language designed for structured data computation, s

GeoServer

Data Pipelines & ETL

Official GeoServer repository

RudderStack

Data Pipelines & ETL

Privacy and Security focused Segment-alternative, in Golang and React

Maestro

Data Pipelines & ETL

Maestro: Netflix’s Workflow Orchestrator

GraphScope

Data Pipelines & ETL

🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计

Nextflow

Data Pipelines & ETL

A workflow language for data-driven computational pipelines

Quary

BI & Dashboards

Open-source BI for engineers

MetricFlow

Data Pipelines & ETL

MetricFlow allows you to define, build, and maintain metrics in code.

Bruin

Data Pipelines & ETL

Build data pipelines with SQL and Python, ingest data from different sources, add quality

Ktx

AI Infrastructure

ktx is an executable context layer for data and analytics agents 🐙 Allow Claude Code, Cod

Duckle

Data Pipelines & ETL

Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-

Apache StreamPipes

Data Pipelines & ETL

Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users

Nussknacker

Low-code & Internal Tools

Low-code tool for automating actions on real time data | Stream processing for the users.

Preswald

BI & Dashboards

Preswald is a WASM packager for Python-based interactive data apps: bundle full complex da

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

pg_mooncake

Databases

Real-time analytics on Postgres tables

Trench

Analytics

Trench — Open-Source Analytics Infrastructure. A single production-ready Docker image buil

Pachyderm

Data Pipelines & ETL

Data-Centric Pipelines and Data Versioning

Azkaban

Data Pipelines & ETL

Azkaban workflow manager.

ToroDB Stampede

Data Pipelines & ETL

The ToroDB solution to provide better analytics on top of MongoDB and make it easier to mi

Dozer

Data Pipelines & ETL

Dozer is a real-time data movement tool that leverages CDC from various sources and moves

Dremio OSS

Data Pipelines & ETL

Dremio - the missing link in modern data

Optimus

Data Pipelines & ETL

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transfo

Sparklens

Monitoring & Observability

Qubole Sparklens tool for performance tuning Apache Spark