Mesh-LLM/mesh-llm

What is it?

Key features

• 01Exposes one OpenAI-compatible /v1 API across all nodes in the mesh.
• 02Routes requests by model name to the local or peer node that serves it.
• 03Splits models too large for one machine into layer stages across nodes.
• 04Encrypts peer traffic end to end with QUIC.
• 05Supports private meshes via invite tokens and public meshes via Nostr discovery.
• 06Includes a web console and a headless mode for servers.

Use cases

• Pooling spare GPUs from several workstations to run a model larger than any one of them.
• Giving a team a shared private LLM endpoint across machines on a LAN.
• Pointing coding agents such as Claude Code or OpenCode at a self-hosted model pool.
• Running a single GGUF model as a standalone OpenAI-compatible server for development.

Quick start

1. 1

   Run the official install script for your OS (see README)

2. 2

   Or install with Homebrew on Apple Silicon Macs

   brew install Mesh-LLM/tap/mesh-llm
3. 3

   Finish setup

   mesh-llm setup
4. 4

   Join the public mesh and start serving on port 9337

   mesh-llm serve --auto

Things to know

• !Native Windows install has known bugs, so the README points to a prerelease build for now.
• !The model "mesh" fan-out (Mixture-of-Agents) feature is experimental and may change.
• !Source builds need several toolchain pieces, including just, cmake, Rust, sccache, a fast linker and Node.js 24.

In collections

Topics

FAQ

What is mesh-llm?

Pools GPUs and memory across machines into one OpenAI-compatible API, routing requests to local, peer, or split-stage models. Mesh LLM combines GPUs and memory from several machines and exposes them as a single OpenAI-compatible endpoint, served by default at localhost:9337/v1. A model runs locally when one node can host it, is routed by its model field to a peer that can, or is split into layer stages across nodes when it is too large for any single machine.

Is mesh-llm free to use? What license does it have?

mesh-llm is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.

Is mesh-llm still maintained?

Actively maintained. The last commit to mesh-llm was on Nov 6, 2026.

What can I use mesh-llm for?

Pooling spare GPUs from several workstations to run a model larger than any one of them. Giving a team a shared private LLM endpoint across machines on a LAN. Pointing coding agents such as Claude Code or OpenCode at a self-hosted model pool. Running a single GGUF model as a standalone OpenAI-compatible server for development.

How do I install mesh-llm?

Run the official install script for your OS (see README) Or install with Homebrew on Apple Silicon Macs: brew install Mesh-LLM/tap/mesh-llm Finish setup: mesh-llm setup Join the public mesh and start serving on port 9337: mesh-llm serve --auto

Where can I get mesh-llm?

The source code and installation instructions are on GitHub: https://github.com/Mesh-LLM/mesh-llm