Gobblin

Open source

Apache Gobblin is a distributed data integration framework for ingestion, replication and lifecycle management of streaming and batch data.

Open-source alternative to

gobblin.apache.org
Gobblin homepage screenshot
GitHub stars
2.3k
Last 30 days
+1(+0%)
Last commit
29 days ago
Repository age
11 years
Version
gobblin_0.11.0
Licence
Apache-2.0
Self-hosted
Yes

About Gobblin

Apache Gobblin is a distributed framework for big data integration. It simplifies common work such as data ingestion, replication, organisation and lifecycle management, for both streaming and batch data ecosystems. It became an Apache Top Level Project in January 2021.

It can run in several execution modes. Standalone mode runs as a single application on one machine and also supports an embedded mode. MapReduce mode runs as a MapReduce application on multiple Hadoop versions, with Azkaban supported for launching jobs. A cluster mode using YARN has primary and worker nodes and supports high availability.

A cloud mode runs as an elastic cluster on a public cloud, also with high availability. The news list on the site shows releases from 0.13.0-incubating in 2018 up to 0.17.0 in August 2023, and the project is published by the Apache Software Foundation under the Apache licence.

Key features

  • Data ingestion, replication and lifecycle management
  • Handles both streaming and batch data
  • Standalone mode on a single machine
  • MapReduce mode across Hadoop versions
  • Elastic cluster mode on public cloud

Good fit for

  • Ingesting data into Hadoop or cloud stores
  • Replicating datasets
  • Managing data lifecycle
Built with
Java tools
Tags
data-ingestion
big-data
apache
hadoop
batch
streaming
data-integration

Gobblin: questions and answers

What is Gobblin used for?
Gobblin is a distributed data integration framework for ingestion, replication and lifecycle management of streaming and batch data. It is a good fit for ingesting data into Hadoop or cloud stores, replicating datasets and managing data lifecycle.
Is Gobblin open source?
Yes. Gobblin is open source under the Apache-2.0 licence. Its source code is on GitHub at apache/gobblin and is written mainly in Java.
Is Gobblin free?
Yes. Gobblin is open source, so the software itself is free to use.
Can I self-host Gobblin?
Yes. Gobblin can be self-hosted on your own server or infrastructure.
What is Gobblin an alternative to?
Gobblin is an open-source alternative to Google Cloud Dataflow. Other open-source alternatives to Google Cloud Dataflow include Apache Spark, Kafka and Airflow.
Is Gobblin actively maintained?
Yes. The most recent commit to Gobblin was on 15 October 2026. The project has 2.3k stars on GitHub, with 1 added in the last 30 days.

Open-source alternatives to Gobblin

See all

SaaS alternatives to Gobblin

See all