Repolode
Submit
ArroyoSystems

ArroyoSystems/arroyo

A distributed stream processing engine written in Rust that runs stateful SQL pipelines over real-time data streams.

Open-source alternative to:Apache Flink

View on GitHubWebsite
Stars
5,062
Forks
380
Open issues
130
Last commit
yesterday

What is it?

Arroyo is a stream processing engine that lets you write SQL queries over continuously arriving data and get results as soon as they are available, rather than waiting for a batch job to finish. It supports stateful operations such as windows and joins, and checkpoints state so pipelines can recover from failures and be rescheduled.

Arroyo ships as a single binary. You can start a local cluster with the CLI or run it in Docker, then manage pipelines through a web UI listening on port 5115. A wide range of connectors is available, including Kafka and Iceberg, and the project is designed to run in cloud environments with seamless scaling.

The project is aimed at teams that want real-time analytics or data ingestion without deep streaming expertise, since pipeline APIs are separated from the internal implementation. Cloudflare also offers a managed version of Arroyo as Cloudflare Pipelines, currently for stateless pipelines writing to R2.

Key features

  • 01Write streaming pipelines in SQL
  • 02Stateful windows and joins
  • 03Checkpointing for fault tolerance and recovery
  • 04Event-time processing based on the Dataflow model
  • 05Connectors including Kafka and Iceberg
  • 06Single binary with CLI and web UI

Use cases

  • Detecting fraud or security incidents in live event streams
  • Building real-time product or business dashboards
  • Ingesting streaming data into a data warehouse or data lake
  • Generating real-time features for machine learning models

Quick start

  1. 1

    Install on macOS with Homebrew

    brew install arroyosystems/tap/arroyo
  2. 2

    Start a local cluster

    arroyo cluster
  3. 3

    Open the Web UI at http://localhost:5115 once the cluster is running

Things to know

  • Self-hosting requires running the Arroyo cluster and its components yourself.
  • Stateful pipelines are not yet available on the managed Cloudflare Pipelines service, which currently supports only stateless pipelines into R2.

In collections

Databases & SQL toolsMicroservice frameworks & service infrastructure

Topics

#stream-processing#sql#rust#data-streaming#kafka#real-time-analytics#distributed-systems#data#data-stream-processing#dev-tools

FAQ

What is arroyo?
A distributed stream processing engine written in Rust that runs stateful SQL pipelines over real-time data streams. Arroyo is a stream processing engine that lets you write SQL queries over continuously arriving data and get results as soon as they are available, rather than waiting for a batch job to finish. It supports stateful operations such as windows and joins, and checkpoints state so pipelines can recover from failures and be rescheduled.
Is arroyo free to use? What license does it have?
arroyo is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is arroyo still maintained?
Actively maintained. The last commit to arroyo was on Oct 23, 2026.
What can I use arroyo for?
Detecting fraud or security incidents in live event streams Building real-time product or business dashboards Ingesting streaming data into a data warehouse or data lake Generating real-time features for machine learning models
How do I install arroyo?
Install on macOS with Homebrew: brew install arroyosystems/tap/arroyo Start a local cluster: arroyo cluster Open the Web UI at http://localhost:5115 once the cluster is running
Where can I get arroyo?
The source code and installation instructions are on GitHub: https://github.com/ArroyoSystems/arroyo

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects