Repolode
Submit
bacalhau-project

bacalhau-project/bacalhau

Open-source framework for running compute jobs where data lives, instead of moving large datasets across networks.

View on GitHubWebsite
Stars
874
Forks
105
Open issues
5
Last commit
2 days ago

What is it?

Bacalhau is an open-source distributed compute orchestration framework built around the idea of bringing compute to the data. Rather than transferring large datasets across networks, it runs jobs close to where the data is created or stored, which reduces latency, bandwidth use and egress costs.

The project ships as a single self-contained binary that acts as client, orchestrator and compute node. Jobs can run in Docker or WebAssembly engines, read from storage such as S3, HTTP/HTTPS, IPFS or local volumes, and publish results back to those backends. Jobs can be defined declaratively in YAML or submitted imperatively from the CLI, and support batch, ops, daemon and service job types.

Bacalhau is aimed at workloads where data is large, sensitive or distributed, such as log processing, multi-region analytics, and edge deployments, and where data must stay within its own security boundary.

Key features

  • 01Single binary that works as client, orchestrator and compute node
  • 02Runs jobs in Docker or WebAssembly execution engines
  • 03Reads from S3, HTTP/HTTPS, IPFS and local storage
  • 04Supports batch, ops, daemon and service job types
  • 05Accepts jobs as declarative YAML or imperative CLI arguments

Use cases

  • Processing logs at the source instead of shipping them to a central cluster
  • Querying or analyzing data spread across several regions
  • Running ML training or inference close to where data is produced
  • Executing vetted computations on sensitive data that cannot leave its premises

Quick start

  1. 1

    Install the Bacalhau CLI (Linux/macOS)

    Install with the official instructions in the README
  2. 2

    Verify the installation

    bacalhau version

Things to know

  • The quick install script targets Linux and macOS only.
  • The README points to external docs for running your first job, so setup beyond installation is not shown here.

Topics

#distributed-computing#orchestration#data-processing#docker#webassembly#edge-computing#batch-processing#ai-art#ai-data-collection#ai-pipeline

FAQ

What is bacalhau?
Open-source framework for running compute jobs where data lives, instead of moving large datasets across networks. Bacalhau is an open-source distributed compute orchestration framework built around the idea of bringing compute to the data. Rather than transferring large datasets across networks, it runs jobs close to where the data is created or stored, which reduces latency, bandwidth use and egress costs.
Is bacalhau free to use? What license does it have?
bacalhau is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is bacalhau still maintained?
Actively maintained. The last commit to bacalhau was on Oct 18, 2026.
What can I use bacalhau for?
Processing logs at the source instead of shipping them to a central cluster Querying or analyzing data spread across several regions Running ML training or inference close to where data is produced Executing vetted computations on sensitive data that cannot leave its premises
How do I install bacalhau?
Install the Bacalhau CLI (Linux/macOS) Verify the installation: bacalhau version
Where can I get bacalhau?
The source code and installation instructions are on GitHub: https://github.com/bacalhau-project/bacalhau

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects