Repolode
Submit
ddalcu

ddalcu/mlx-serve

Native local LLM and media server for Apple Silicon Macs, with OpenAI, Anthropic and Ollama compatible APIs and no Python required.

Open-source alternative to:OllamaLM Studio

View on GitHubWebsite
Stars
2,390
Forks
252
Open issues
76
Last commit
2 days ago

What is it?

MLX-Serve runs open models on a Mac through one native macOS app and one local server. The server exposes OpenAI chat, completions and Responses endpoints, Anthropic Messages, and Ollama APIs on a single port at localhost:11234, so existing clients and coding agents such as Claude Code can point at it without changes. The core is written in Zig and runs on Apple's MLX framework, with no Python in the runtime or build.

Beyond text chat, the app and HTTP API cover image, video, music, speech and 3D generation, including speech-to-text transcription and voice cloning. Models are downloaded through a CLI with commands like run, pull and serve, and the app suggests a model sized to the Mac's memory. Larger mixture-of-experts models can be streamed from SSD on 128 GB machines, and two Macs connected by Thunderbolt 5 can run one model together.

The project also ships an iPhone app for on-device chat and media generation, and Linux builds for NVIDIA GPUs. It is MIT licensed and builds from source with Xcode and Homebrew.

Key features

  • 01Serves OpenAI, Anthropic and Ollama APIs on one local port with streaming and tool calls.
  • 02Runs chat, image, video, music, speech and 3D models natively on Apple Silicon via MLX.
  • 03Includes a macOS app with a built-in web console and automatic model suggestions by memory size.
  • 04Provides a CLI for downloading, running and serving models, with resumable pulls.
  • 05Supports Gemma, Qwen, GLM and DeepSeek model families, plus any MLX or GGUF model with a supported architecture.
  • 06Offers Claude Code and other coding-agent integrations pointed at the local model.

Use cases

  • Running a private, offline chat assistant on a Mac without sending data to a cloud API.
  • Pointing existing OpenAI or Anthropic SDK code at a local model during development.
  • Using Claude Code or other coding agents against a locally hosted model.
  • Generating images, speech or music on a Mac or iPhone without a cloud service.

Quick start

  1. 1

    Tap the Homebrew repository

    brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serve
  2. 2

    Trust the tap

    brew trust ddalcu/mlx-serve
  3. 3

    Install the command-line server

    brew install mlx-serve
  4. 4

    Download Gemma 4 and chat with it in the terminal

    mlx-serve run gemma4

Things to know

  • Runs only on Apple Silicon Macs; the Mac app requires macOS 26.2 or later.
  • Large models such as the 128 GB and 256 GB tiers need a lot of memory and stream weights from the SSD.

In collections

Claude Code tools, skills & pluginsRun LLMs locallyOpen-source Mac apps

Topics

#apple-silicon#local-llm#mlx#openai-api#anthropic-api#inference#macos#tool-calling#agent#claude-code

FAQ

What is mlx-serve?
Native local LLM and media server for Apple Silicon Macs, with OpenAI, Anthropic and Ollama compatible APIs and no Python required. MLX-Serve runs open models on a Mac through one native macOS app and one local server. The server exposes OpenAI chat, completions and Responses endpoints, Anthropic Messages, and Ollama APIs on a single port at localhost:11234, so existing clients and coding agents such as Claude Code can point at it without changes. The core is written in Zig and runs on Apple's MLX framework, with no Python in the runtime or build.
Is mlx-serve free to use? What license does it have?
mlx-serve is open source under the Other license. Check the license terms before using it in commercial products.
Is mlx-serve still maintained?
Actively maintained. The last commit to mlx-serve was on Nov 6, 2026.
What can I use mlx-serve for?
Running a private, offline chat assistant on a Mac without sending data to a cloud API. Pointing existing OpenAI or Anthropic SDK code at a local model during development. Using Claude Code or other coding agents against a locally hosted model. Generating images, speech or music on a Mac or iPhone without a cloud service.
How do I install mlx-serve?
Tap the Homebrew repository: brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serve Trust the tap: brew trust ddalcu/mlx-serve Install the command-line server: brew install mlx-serve Download Gemma 4 and chat with it in the terminal: mlx-serve run gemma4
Where can I get mlx-serve?
The source code and installation instructions are on GitHub: https://github.com/ddalcu/mlx-serve

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects