Repolode
Submit
warpstreamlabs

warpstreamlabs/bento

A Go stream processor that connects message brokers, cloud services and databases, with a mapping language for transforming payloads in between.

View on GitHubWebsite
Stars
2,168
Forks
236
Open issues
153
Last commit
6 minutes ago

What is it?

Bento is a stream processor written in Go. It reads data from sources such as Kafka, NATS, Redis streams, AWS and GCP services, and SQL databases, runs it through a chain of processors, and writes it to sinks. Pipelines are declared in YAML, so a basic setup can fit in a single config file.

Processing stages include enrichment, transformation and filtering, and payloads are reshaped with Bloblang, Bento's built-in mapping language. Delivery is handled by an in-process transaction model without on-disk state, so with at-least-once sources and sinks it aims to preserve at-least-once delivery across crashes. Bento ships as a static binary, a Docker image, or a serverless function.

It exposes liveness and readiness endpoints and emits metrics to Statsd, Prometheus and other backends, plus OpenTelemetry traces. Custom components can be added as Go plugins. This repository is a fork of the original project made before its license change.

Key features

  • 01Connects to Kafka, NATS, Redis, AWS, GCP, Azure, SQL databases and more as inputs and outputs.
  • 02Declarative YAML pipelines with a built-in mapping language, Bloblang.
  • 03Delivers at-least-once without on-disk state when sources and sinks support it.
  • 04Ships as a static binary, Docker image, or serverless function.
  • 05Provides /ping and /ready health endpoints plus Prometheus, Statsd and OpenTelemetry observability.
  • 06Supports custom components written as Go plugins.

Use cases

  • Moving events from Kafka or NATS into a database or search index with light transformation.
  • Enriching or filtering a message stream before it reaches downstream services.
  • Bridging cloud services such as AWS Kinesis or GCP Pub/Sub with other brokers.
  • Running a lightweight containerized pipeline without operating a heavier stream platform.

Quick start

  1. 1

    Pull the Docker image

    docker pull ghcr.io/warpstreamlabs/bento
  2. 2

    Run Bento in Docker with your config file mounted

    docker run --rm -v /path/to/your/config.yaml:/bento.yaml ghcr.io/warpstreamlabs/bento

Things to know

  • The README says the release process is still being worked on, so there is no standard binary install yet.
  • The license is not clearly identified in the project metadata.
  • Some components need external C libraries and a special build tag, such as zmq4.

Topics

#stream-processing#data-pipeline#kafka#go#etl#message-broker#cloud-native#benthos#bento#connect

FAQ

What is bento?
A Go stream processor that connects message brokers, cloud services and databases, with a mapping language for transforming payloads in between. Bento is a stream processor written in Go. It reads data from sources such as Kafka, NATS, Redis streams, AWS and GCP services, and SQL databases, runs it through a chain of processors, and writes it to sinks. Pipelines are declared in YAML, so a basic setup can fit in a single config file.
Is bento free to use? What license does it have?
bento is open source under the Other license. Check the license terms before using it in commercial products.
Is bento still maintained?
Actively maintained. The last commit to bento was on Oct 22, 2026.
What can I use bento for?
Moving events from Kafka or NATS into a database or search index with light transformation. Enriching or filtering a message stream before it reaches downstream services. Bridging cloud services such as AWS Kinesis or GCP Pub/Sub with other brokers. Running a lightweight containerized pipeline without operating a heavier stream platform.
How do I install bento?
Pull the Docker image: docker pull ghcr.io/warpstreamlabs/bento Run Bento in Docker with your config file mounted: docker run --rm -v /path/to/your/config.yaml:/bento.yaml ghcr.io/warpstreamlabs/bento
Where can I get bento?
The source code and installation instructions are on GitHub: https://github.com/warpstreamlabs/bento

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects