Repolode
Submit
Generated-Computers

Generated-Computers/Locals

A local inference server for macOS Apple silicon that serves many AI coding agents at once, with batched decoding and grammar-constrained tool calls.

View on GitHubWebsite
Stars
635
Forks
40
Open issues
7
Last commit
8 hours ago

What is it?

Locals is a local model server for Mac that is built for agents rather than plain chat. Agents send tool calls, long conversations and many parallel requests, and Locals handles these with batched GPU kernels, tool-call output constrained by each tool's schema, and a prompt cache that is kept across turns and restarts. The README reports up to 3x the throughput of mlx-lm when eight requests run at once on an M2 Pro with 16 GB, using Qwen3.5-4B.

It exposes OpenAI, Anthropic and Ollama compatible APIs, so tools such as OpenCode, Cline, Aider, OpenHands, Claude Code and Hermes Agent can connect with a single launch command, for example `locals code ~/my-project`. Locals also includes a `locals run` mode that splits a task across several agents, each working in its own copy of the project and checked by the project's tests. Everything runs on the local machine, and the project requires Apple silicon with macOS 14 or newer.

Key features

  • 01Batched decoding so several agent requests share one pass on the GPU.
  • 02Tool calls constrained by the tool's schema, so arguments match their types.
  • 03Prompt and conversation caching that persists across turns and restarts.
  • 04Compatible with OpenAI, Anthropic and Ollama APIs.
  • 05One-command launch for OpenCode, Aider, Cline, OpenHands, Claude Code and more.
  • 06Multi-agent `locals run` mode that tests each part in an isolated project copy.

Use cases

  • Running several coding agents in parallel on a Mac without sending code to a cloud API.
  • Working on a local project with OpenCode or Aider using a model that runs on the laptop.
  • Serving a local OpenAI-compatible endpoint to editor extensions such as Cline or Roo Code.
  • Asking questions about a large codebase or document with a long, cached context.

Quick start

  1. 1

    Run the official install script from the README

  2. 2

    Or install from PyPI with uv

    uv tool install --python 3.12 locals
  3. 3

    In a new Terminal window, open OpenCode in your project

    locals code ~/my-project

Things to know

  • Runs only on macOS with Apple silicon.
  • Flint needs 24 GB or more of memory, and the newest Flint release is not yet on PyPI.
  • Some measured chips (M1, M3, M4) have not been benchmarked yet.

In collections

Coding with local LLMsOpen-source AI agent frameworks & appsAI coding assistants & agentsRun LLMs locallyOpen-source Mac apps

Topics

#ai-agents#local-llm#apple-silicon#mlx#inference-engine#coding-agents#openai-compatible#macos#agents#artificial-intelligence

FAQ

What is Locals?
A local inference server for macOS Apple silicon that serves many AI coding agents at once, with batched decoding and grammar-constrained tool calls. Locals is a local model server for Mac that is built for agents rather than plain chat. Agents send tool calls, long conversations and many parallel requests, and Locals handles these with batched GPU kernels, tool-call output constrained by each tool's schema, and a prompt cache that is kept across turns and restarts. The README reports up to 3x the throughput of mlx-lm when eight requests run at once on an M2 Pro with 16 GB, using Qwen3.5-4B.
Is Locals free to use? What license does it have?
Locals is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is Locals still maintained?
Actively maintained. The last commit to Locals was on Nov 6, 2026.
What can I use Locals for?
Running several coding agents in parallel on a Mac without sending code to a cloud API. Working on a local project with OpenCode or Aider using a model that runs on the laptop. Serving a local OpenAI-compatible endpoint to editor extensions such as Cline or Roo Code. Asking questions about a large codebase or document with a long, cached context.
How do I install Locals?
Run the official install script from the README Or install from PyPI with uv: uv tool install --python 3.12 locals In a new Terminal window, open OpenCode in your project: locals code ~/my-project
Where can I get Locals?
The source code and installation instructions are on GitHub: https://github.com/Generated-Computers/Locals

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects