Repolode
Submit
lmstudio-ai

lmstudio-ai/mlx-engine

LLM inference engine that runs MLX models on Apple silicon, used as the backend for LM Studio.

View on GitHub
Stars
1,200
Forks
138
Open issues
97
Last commit
4 days ago

What is it?

mlx-engine is the Apple MLX-based LLM engine that LM Studio uses to load and run models on Mac. It wraps mlx-lm for text generation, Outlines for structured output, and mlx-vlm for vision models, so LM Studio users can run MLX-format models locally on Apple silicon.

The repository includes a standalone demo script, demo.py, that loads an MLX text model or vision model and generates a response from a prompt, with optional images and a draft model for speculative decoding. Models are downloaded with the lms CLI from LM Studio. Supported vision models include Llama-3.2-Vision, Pixtral, Qwen2-VL and Llava-v1.6.

The project requires macOS 14 or later and Python 3.11, and is released under the MIT license.

Key features

  • 01Runs MLX-format text LLMs on Apple silicon Macs.
  • 02Supports vision models such as Pixtral, Qwen2-VL and Llama-3.2-Vision.
  • 03Provides speculative decoding with a separate draft model.
  • 04Includes structured output support via Outlines.
  • 05Ships as the built-in engine in LM Studio 0.3.4 and newer for Mac.

Use cases

  • Running local LLMs privately on a Mac through LM Studio.
  • Testing MLX text or vision models with the standalone demo script.
  • Building a local Python application on top of MLX model inference.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/lmstudio-ai/mlx-engine.git
  2. 2

    Change into the project directory

    cd mlx-engine
  3. 3

    Install the required dependency packages

    pip install -U -r requirements.txt
  4. 4

    Download a model with the lms CLI and run the demo

    lms get mlx-community/Meta-Llama-3.1-8B-Instruct-4bit

Things to know

  • Requires macOS 14.0 or later on Apple silicon.
  • Standalone demo requires python3.11 specifically.
  • Model downloads are large, several gigabytes each.

In collections

Open-source Mac apps

Topics

#mlx#llm#apple-silicon#inference#vision-model#lm-studio#python

FAQ

What is mlx-engine?
LLM inference engine that runs MLX models on Apple silicon, used as the backend for LM Studio. mlx-engine is the Apple MLX-based LLM engine that LM Studio uses to load and run models on Mac. It wraps mlx-lm for text generation, Outlines for structured output, and mlx-vlm for vision models, so LM Studio users can run MLX-format models locally on Apple silicon.
Is mlx-engine free to use? What license does it have?
mlx-engine is open source under the MIT license. Check the license terms before using it in commercial products.
Is mlx-engine still maintained?
Actively maintained. The last commit to mlx-engine was on Nov 3, 2026.
What can I use mlx-engine for?
Running local LLMs privately on a Mac through LM Studio. Testing MLX text or vision models with the standalone demo script. Building a local Python application on top of MLX model inference.
How do I install mlx-engine?
Clone the repository: git clone https://github.com/lmstudio-ai/mlx-engine.git Change into the project directory: cd mlx-engine Install the required dependency packages: pip install -U -r requirements.txt Download a model with the lms CLI and run the demo: lms get mlx-community/Meta-Llama-3.1-8B-Instruct-4bit
Where can I get mlx-engine?
The source code and installation instructions are on GitHub: https://github.com/lmstudio-ai/mlx-engine

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects