AutoMQ/automq

What is it?

Key features

• 01Stores topic data on S3 or S3-compatible object storage instead of local disks.
• 02Brokers are stateless, allowing scaling in or out within seconds.
• 03Built-in auto-balancer redistributes partitions and traffic without Cruise Control.
• 04Rack-aware routing aims to avoid cross-AZ traffic charges on AWS and GCP.
• 05Exports Prometheus and OpenTelemetry metrics.
• 06Claims full compatibility with the Apache Kafka protocol.

Use cases

• Replacing a self-managed Kafka cluster in the cloud to reduce storage and cross-AZ network costs.
• Running event streaming workloads with frequent scaling against variable traffic.
• Building a Kafka-compatible streaming layer on on-premise S3-compatible storage such as MinIO.
• Evaluating a diskless Kafka deployment locally with the provided Docker Compose setup.

Quick start

1. 1

   Download the single-node Docker Compose file

   curl -O https://raw.githubusercontent.com/AutoMQ/automq/refs/tags/1.5.5/docker/docker-compose.yaml
2. 2

   Start AutoMQ and MinIO with Docker Compose

   docker compose -f docker-compose.yaml up -d
3. 3

   Run a test Kafka producer against AutoMQ

   docker run --network automq_net automqinc/automq:latest /bin/bash -c "/opt/automq/kafka/bin/kafka-producer-perf-test.sh --topic test-topic --num-records=1024000 --throughput 5120 --record-size 1024 --producer-props bootstrap.servers=server1:9092 linger.ms=100 batch.size=524288 buffer.memory=134217728 max.request.size=67108864"
4. 4

   Stop and remove the setup when finished

   docker compose -f docker-compose.yaml down

Things to know

• !The Quick Start is for evaluation only and is not suitable for production use.
• !Requires Docker, at least 4 GB RAM for Docker, and ports 9092 and 9000 free.
• !Default latency is hundreds of milliseconds because it writes directly to S3; low latency is in the enterprise version.

Topics

FAQ

What is automq?

A stateless, Kafka-compatible streaming system that stores data on S3 or S3-compatible object storage instead of local disks. AutoMQ is a fork of Apache Kafka that replaces its local-disk storage engine with one built on object storage such as Amazon S3 or MinIO. Brokers hold no persistent state, so partitions and traffic can be moved between nodes without copying data, which the project says allows scaling in or out within seconds.

Is automq free to use? What license does it have?

automq is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.

Is automq still maintained?

Actively maintained. The last commit to automq was on Oct 19, 2026.

What can I use automq for?

Replacing a self-managed Kafka cluster in the cloud to reduce storage and cross-AZ network costs. Running event streaming workloads with frequent scaling against variable traffic. Building a Kafka-compatible streaming layer on on-premise S3-compatible storage such as MinIO. Evaluating a diskless Kafka deployment locally with the provided Docker Compose setup.

How do I install automq?

Download the single-node Docker Compose file: curl -O https://raw.githubusercontent.com/AutoMQ/automq/refs/tags/1.5.5/docker/docker-compose.yaml Start AutoMQ and MinIO with Docker Compose: docker compose -f docker-compose.yaml up -d Run a test Kafka producer against AutoMQ: docker run --network automq_net automqinc/automq:latest /bin/bash -c "/opt/automq/kafka/bin/kafka-producer-perf-test.sh --topic test-topic --num-records=1024000 --throughput 5120 --record-size 1024 --producer-props bootstrap.servers=server1:9092 linger.ms=100 batch.size=524288 buffer.memory=134217728 max.request.size=67108864" Stop and remove the setup when finished: docker compose -f docker-compose.yaml down

Where can I get automq?

The source code and installation instructions are on GitHub: https://github.com/AutoMQ/automq