About PipelineDB
PipelineDB was a PostgreSQL extension designed for high-performance time-series aggregation, built to power real-time reporting and analytics applications. It let developers define continuous SQL queries that perpetually aggregated incoming time-series data, storing only the aggregated output in regular, queryable tables rather than the raw data itself.
This approach worked like an extremely high-throughput materialized view that updated incrementally and never needed to be manually refreshed, since raw time-series data was never written to disk, making it efficient for aggregation-heavy workloads. Continuous queries produced their own output streams, so they could be chained together into arbitrary networks of continuous SQL.
The README states that PipelineDB joined Confluent and that the project would receive no new releases beyond 1.0.0, though critical bugs would still be fixed. It ran on 64-bit systems with PostgreSQL 10 and 11, required ZeroMQ for inter-process communication, and was written in C and released under the Apache-2.0 license. New projects should look at actively maintained streaming or aggregation tools instead.
Key features
- Continuous SQL aggregation queries
- Incrementally updated materialized views
- No raw time-series data written to disk
- Chainable continuous query output streams
- PostgreSQL extension architecture
Good fit for
- Real-time analytics dashboards on PostgreSQL
- High-throughput time-series aggregation
PipelineDB: questions and answers
- What is PipelineDB used for?
- PipelineDB was an Apache-licensed PostgreSQL extension for high-performance, continuous time-series aggregation, now discontinued after its team joined Confluent. It is a good fit for real-time analytics dashboards on PostgreSQL and high-throughput time-series aggregation.
- Is PipelineDB open source?
- Yes. PipelineDB is open source under the Apache-2.0 licence. Its source code is on GitHub at pipelinedb/pipelinedb and is written mainly in C.
- Is PipelineDB free?
- Yes. PipelineDB is open source, so the software itself is free to use.
- Can I self-host PipelineDB?
- Yes. PipelineDB can be self-hosted on your own server or infrastructure; there is no official hosted version.
- What is PipelineDB an alternative to?
- PipelineDB is an open-source alternative to Tinybird. Other open-source alternatives to Tinybird include TimescaleDB, QuestDB and CrateDB.
- Is PipelineDB actively maintained?
- The most recent commit to PipelineDB was on 20 February 2022, and the latest release is 1.0.0-13, published on 19 February 2019. The project has 2.7k stars on GitHub, with 1 added in the last 30 days.
Open-source alternatives to PipelineDB
See all
TimescaleDB
Databases
A time-series database for high-performance real-time analytics packaged as a Postgres ext
OSSvs KX kdb+★ 24k
QuestDB
Databases
QuestDB is a high performance, open-source, time-series database
Apache-2.0vs Tinybird★ 18k
CrateDB
Databases
CrateDB is a distributed and scalable SQL database for storing and analyzing massive amoun
Apache-2.0vs SingleStore★ 4.5k
DuckDB
Databases
DuckDB is an analytical in-process SQL database management system
MITvs MotherDuck★ 43k
ClickHouse
Databases
ClickHouse is a fast open-source column-oriented database management system that allows ge
Apache-2.0vs Snowflake★ 51k
InfluxDB
Databases
Scalable datastore for metrics, events, and real-time analytics
Apache-2.0vs KX kdb+★ 32k
SaaS alternatives to PipelineDB
See all
Tinybird
Databases
Real-time analytics database that turns SQL queries into API endpoints
SaaS
Azure Data Explorer
Databases
Analytics service on Azure for exploring log, telemetry and time-series data with Kusto queries
SaaS
Google BigQuery
Databases
Serverless data warehouse for SQL analytics on large datasets
SaaS
KX kdb+
Databases
Time-series database used for high-frequency market and sensor data analytics
SaaS
MotherDuck
Databases
Managed cloud service for DuckDB analytics
SaaS
Snowflake
Databases
Cloud data platform for warehousing, data sharing and analytics
SaaS
