Repolode
Submit
apache

apache/kafka

Apache Kafka is an open-source distributed event streaming platform for building high-throughput data pipelines and streaming applications.

Open-source alternative to:Amazon KinesisGoogle Cloud Pub/Sub

View on GitHubWebsite
Stars
34,186
Forks
15,663
Open issues
667
Last commit
21 hours ago

What is it?

Apache Kafka is a distributed event streaming platform used for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. It is written in Java and Scala, and runs as a cluster of brokers that store and serve event streams.

The repository contains the full broker and client code base. Developers can build a JAR with Gradle, run a single broker from compiled scripts or a Docker image, and run unit, integration, and system tests. It also includes the Streams and clients modules, Trogdor for benchmarks and workloads, and JMH microbenchmarks.

The build targets Java 17 and 25, with Scala 2.13 as the only supported Scala version. Contributors can run Checkstyle, Spotless, and SpotBugs checks locally before submitting changes.

Key features

  • 01Distributed event streaming with brokers that store and serve event logs
  • 02Client libraries and Kafka Streams modules for producing and consuming events
  • 03Gradle-based build with unit, integration, and system test targets
  • 04Docker image and compiled scripts for running a broker locally
  • 05Code quality tooling including Checkstyle, Spotless, and SpotBugs

Use cases

  • Building real-time data pipelines that move events between services
  • Running streaming analytics on continuously arriving data
  • Contributing to or debugging the Kafka broker and client code
  • Benchmarking streaming workloads with the included Trogdor framework

Quick start

  1. 1

    Install Java (JDK 17 or 25 is used for builds)

  2. 2

    Build the JAR with Gradle

    ./gradlew jar
  3. 3

    Run a Kafka broker using the Docker image

    docker run -p 9092:9092 apache/kafka:latest

Things to know

  • Building from source requires a Java installation and the Gradle build.

In collections

Streaming toolsMicroservice frameworks & service infrastructure

Topics

#event-streaming#distributed-systems#message-queue#java#scala#data-pipeline#kafka#streaming

FAQ

What is kafka?
Apache Kafka is an open-source distributed event streaming platform for building high-throughput data pipelines and streaming applications. Apache Kafka is a distributed event streaming platform used for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. It is written in Java and Scala, and runs as a cluster of brokers that store and serve event streams.
Is kafka free to use? What license does it have?
kafka is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is kafka still maintained?
Actively maintained. The last commit to kafka was on Nov 6, 2026.
What can I use kafka for?
Building real-time data pipelines that move events between services Running streaming analytics on continuously arriving data Contributing to or debugging the Kafka broker and client code Benchmarking streaming workloads with the included Trogdor framework
How do I install kafka?
Install Java (JDK 17 or 25 is used for builds) Build the JAR with Gradle: ./gradlew jar Run a Kafka broker using the Docker image: docker run -p 9092:9092 apache/kafka:latest
Where can I get kafka?
The source code and installation instructions are on GitHub: https://github.com/apache/kafka

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects