Open-source Amazon Kinesis alternatives

Apache Kafka is a platform for distributed event streaming. Teams use it to move and process streams of events for data pipelines, streaming analytics, data integration and business-critical applications. The code lives in the Apache Software Foundation repository, is written mainly in Java with Scala for parts of the broker, and is released under the Apache-2.0 license.

Kafka is installed software that you run yourself. The project builds and tests against recent Java versions, supports Scala 2.13 only, and documents how to build a JAR and start a broker through its quickstart. The repository also contains client and streams modules, along with test tooling for unit and integration runs. Because Kafka is operated as infrastructure, adopters typically run clusters on their own servers or on Kubernetes, though managed services from other vendors exist.

RisingWave is an event streaming platform, positioned for agentic AI and real-time applications, that continuously takes in data, transforms it and serves it. It is meant to replace the familiar chain of Debezium (change data capture), Kafka (transport), Flink (processing) and a serving database with one system, removing the latency and operational work of each hop.

Sources include webhooks from SaaS applications, native change data capture from PostgreSQL and MySQL, event streams such as Kafka, Pulsar and Kinesis, and batch data from S3 and warehouses, all unified under a SQL interface where streams and tables can be joined. Processing is incremental: if upstream data changes, just the affected results get recomputed, which keeps materialized views up to date without full recomputation. Results are queryable directly, and the topics mention PostgreSQL compatibility and Apache Iceberg.

RisingWave is written in Rust and licensed under Apache-2.0. It can be tried with Docker or Kubernetes using the quick start guide, and it suits data engineers who want streaming pipelines defined in SQL instead of several separate services.