Repolode
投稿する
AutoMQ

AutoMQ/automq

A stateless, Kafka-compatible streaming system that stores data on S3 or S3-compatible object storage instead of local disks.

代替できる製品:Amazon MSKConfluent Cloud

GitHub で見る公式サイト
スター
10,917
フォーク
797
未解決の Issue
66
最終コミット
4 時間前

これは何？

AutoMQ is a fork of Apache Kafka that replaces its local-disk storage layer with object storage such as Amazon S3 or MinIO. Brokers hold no persistent state, so partitions do not need to be moved when a cluster scales in or out, and the project states that scaling takes seconds.

The project is aimed at teams running Kafka in the cloud who face high EBS storage bills, cross-AZ traffic charges, and over-provisioning. A write-ahead log (WAL) and caching layers (LogCache, BlockCache) keep write and read paths efficient while data lives in object storage. A built-in auto-balancer moves traffic and partitions between brokers, so a separate tool such as Cruise Control is not needed. A rack-aware router directs clients in each availability zone to local partition metadata to avoid cross-AZ fees on AWS and GCP.

AutoMQ is 100% compatible with the Kafka protocol and exports Prometheus and OpenTelemetry metrics. The open-source edition defaults to S3 directly, which the project describes as adding hundreds of milliseconds of latency; single-digit millisecond latency is offered in the enterprise edition. Docker Compose files are provided for single-node and three-node evaluation setups.

主な機能

  • 01Kafka protocol compatible, with Kafka's storage layer replaced by S3-compatible object storage.
  • 02Stateless brokers that scale in and out without partition data movement.
  • 03Built-in auto-balancer that redistributes partitions and traffic between brokers.
  • 04Rack-aware routing to avoid cross-AZ traffic charges on AWS and GCP.
  • 05Prometheus and OpenTelemetry metrics export.
  • 06Docker Compose setups for single-node and three-node evaluation.

ユースケース

  • Replacing a self-managed Apache Kafka cluster in the cloud to reduce storage and cross-AZ costs.
  • Running streaming workloads that need frequent scaling with variable traffic.
  • Using Kafka clients with an S3-compatible backend on-premise, such as MinIO.
  • Evaluating Kafka-compatible streaming on a local machine before a production deployment.

トピック

#kafka#streaming#object-storage#s3#message-queue#cloud-native#java#automq#diskless#serverless

よくある質問

automq とは？
A stateless, Kafka-compatible streaming system that stores data on S3 or S3-compatible object storage instead of local disks. AutoMQ is a fork of Apache Kafka that replaces its local-disk storage layer with object storage such as Amazon S3 or MinIO. Brokers hold no persistent state, so partitions do not need to be moved when a cluster scales in or out, and the project states that scaling takes seconds.
automq は無料で使えますか？ライセンスは？
automq は Apache-2.0 ライセンスのオープンソースです。商用製品で使う前にライセンス条件を確認してください。
automq は今もメンテナンスされていますか？
活発にメンテナンス中。automq の最終コミットは 2026年10月19日 です。
automq は何に使えますか？
Replacing a self-managed Apache Kafka cluster in the cloud to reduce storage and cross-AZ costs. Running streaming workloads that need frequent scaling with variable traffic. Using Kafka clients with an S3-compatible backend on-premise, such as MinIO. Evaluating Kafka-compatible streaming on a local machine before a production deployment.
automq はどこで入手できますか？
ソースコードとインストール手順は GitHub にあります：https://github.com/AutoMQ/automq

概要はプロジェクトの README をもとに AI が生成しています。GitHub の数値は毎日更新されます。

関連プロジェクト