Repolode
Submit
exo-explore

exo-explore/exo

Pools the devices you own into one AI cluster so large models can run locally across multiple Macs and other machines.

Open-source alternative to:Ollama

View on GitHub
Stars
47,970
Forks
3,568
Open issues
378
Last commit
7 days ago

What is it?

exo connects devices on your network into a single cluster that runs AI models locally. It can split a model that is too large for one machine across several devices, and it discovers peers automatically without manual configuration. On Macs with Thunderbolt 5, it supports RDMA over Thunderbolt to reduce latency between devices, and tensor parallelism to speed up inference as more devices are added.

The project uses MLX as its inference backend and MLX distributed for communication. The cluster exposes an API compatible with the OpenAI Chat Completions, Claude Messages, OpenAI Responses, and Ollama formats, so existing clients and tools can connect to it. A built-in dashboard lets you manage the cluster and chat with loaded models, and custom models can be added from the Hugging Face hub.

The macOS app and running from source are both supported on macOS, while Linux currently runs inference on CPU only.

Key features

  • 01Automatically discovers other exo devices on the network without manual setup.
  • 02Splits models across devices using topology-aware parallelism.
  • 03Supports tensor parallelism and RDMA over Thunderbolt 5 on compatible Macs.
  • 04Exposes OpenAI, Claude, OpenAI Responses, and Ollama compatible APIs.
  • 05Includes a dashboard for managing the cluster and chatting with models.
  • 06Loads custom models from the Hugging Face hub.

Use cases

  • Running a model too large for one Mac by pooling several Macs together.
  • Keeping inference local and private on hardware you already own.
  • Pointing existing OpenAI- or Ollama-compatible tools at a local cluster.
  • Running a coordinator-only node on a machine with good networking but little GPU capacity.

Quick start

  1. 1

    Install the macOS app with Homebrew

    brew install --cask exo
  2. 2

    Open the dashboard and API in your browser

Things to know

  • The macOS app requires macOS Tahoe 26.2 or later.
  • On Linux, exo currently runs on CPU only; GPU support is under development.
  • RDMA over Thunderbolt 5 needs specific Macs, TB5 cables, and matching OS versions on every device.

Topics

#llm#distributed-inference#mlx#local-llm#apple-silicon#rdma#openai-compatible

FAQ

What is exo?
Pools the devices you own into one AI cluster so large models can run locally across multiple Macs and other machines. exo connects devices on your network into a single cluster that runs AI models locally. It can split a model that is too large for one machine across several devices, and it discovers peers automatically without manual configuration. On Macs with Thunderbolt 5, it supports RDMA over Thunderbolt to reduce latency between devices, and tensor parallelism to speed up inference as more devices are added.
Is exo free to use? What license does it have?
exo is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is exo still maintained?
Actively maintained. The last commit to exo was on Oct 13, 2026.
What can I use exo for?
Running a model too large for one Mac by pooling several Macs together. Keeping inference local and private on hardware you already own. Pointing existing OpenAI- or Ollama-compatible tools at a local cluster. Running a coordinator-only node on a machine with good networking but little GPU capacity.
How do I install exo?
Install the macOS app with Homebrew: brew install --cask exo Open the dashboard and API in your browser
Where can I get exo?
The source code and installation instructions are on GitHub: https://github.com/exo-explore/exo

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects