Repolode
Submit
incoai

incoai/splash

A local LLM inference engine for Apple silicon Macs that serves coding agents and OpenAI/Anthropic-compatible APIs using speculative decoding and Metal kernels.

Open-source alternative to:LM Studio

View on GitHubWebsite
Stars
1,593
Forks
153
Open issues
70
Last commit
12 hours ago

What is it?

Splash is a local inference engine for Macs with Apple silicon. It serves language models on a single machine and exposes OpenAI Chat Completions, Responses, Completions and Anthropic Messages compatible endpoints, so existing coding agents and applications can point at localhost instead of a cloud service.

The engine pairs DFlash 2 speculative decoding with Metal kernels written for each supported model family, and it plans memory automatically. It also caches reused prefixes, batches concurrent requests, supports vision input, tool calling and JSON Schema output, and includes a built-in chat page. Supported models include Qwen3.8-27B, Qwen3.6-35B-A3B and Qwen3.8-Flash-Next, in GGUF or MLX quantized formats.

Installation is through Homebrew, and the server is started with a single `splash serve` command. Requirements are an Apple M3 or newer chip, macOS 26.4 or later, and at least 36 GB of unified memory for the 4-bit examples.

Key features

  • 01Serves OpenAI and Anthropic compatible APIs with streaming and tool calls.
  • 02Uses DFlash 2 speculative decoding with per-model Metal kernels.
  • 03Automatically plans memory and reuses cached prefixes across requests.
  • 04Supports image and PDF input, plus JSON Schema structured output.
  • 05Includes a built-in chat page and launchers for several coding agents.

Use cases

  • Running coding agents such as opencode or Claude against a model on your own Mac.
  • Developing applications against an OpenAI- or Anthropic-compatible endpoint without cloud API costs.
  • Keeping prompts and code on a local machine instead of sending them to a hosted service.

Quick start

  1. 1

    Install Splash with Homebrew

    brew install incoai/tap/splash
  2. 2

    Start the server and download the model

    splash serve --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M
  3. 3

    Open http://127.0.0.1:8000 in your browser once it prints Ready

Things to know

  • Requires an Apple silicon Mac with an M3 or newer chip and macOS 26.4 or later.
  • The 4-bit examples need at least 36 GB of unified memory; smaller Macs must use smaller GGUF variants.
  • Supports only a limited set of model families, and whole-BF16 and F16 models are not supported.

Topics

#llm-inference#apple-silicon#metal#local-llm#speculative-decoding#coding-agents#openai-compatible#macos

FAQ

What is splash?
A local LLM inference engine for Apple silicon Macs that serves coding agents and OpenAI/Anthropic-compatible APIs using speculative decoding and Metal kernels. Splash is a local inference engine for Macs with Apple silicon. It serves language models on a single machine and exposes OpenAI Chat Completions, Responses, Completions and Anthropic Messages compatible endpoints, so existing coding agents and applications can point at localhost instead of a cloud service.
Is splash free to use? What license does it have?
splash is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is splash still maintained?
Actively maintained. The last commit to splash was on Oct 20, 2026.
What can I use splash for?
Running coding agents such as opencode or Claude against a model on your own Mac. Developing applications against an OpenAI- or Anthropic-compatible endpoint without cloud API costs. Keeping prompts and code on a local machine instead of sending them to a hosted service.
How do I install splash?
Install Splash with Homebrew: brew install incoai/tap/splash Start the server and download the model: splash serve --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M Open http://127.0.0.1:8000 in your browser once it prints Ready
Where can I get splash?
The source code and installation instructions are on GitHub: https://github.com/incoai/splash

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects