A node-graph GUI, API and backend for running diffusion models locally to generate images, video, audio and 3D content.
apple-aiml-research/ml-stable-diffusion
Converts Stable Diffusion PyTorch models to Core ML and runs image generation on Apple Silicon devices via a Python package and a Swift package.
- Stars
- 17,968
- Forks
- 1,080
- Open issues
- 192
- Last commit
- last month
What is it?
ml-stable-diffusion is Apple's toolkit for running Stable Diffusion on Apple Silicon using Core ML. It has two parts: a Python package that converts PyTorch checkpoints from Hugging Face into Core ML model files and generates images with the diffusers library, and a Swift package that can be added to Xcode projects to run those Core ML models inside macOS, iOS or iPadOS apps.
The conversion step supports Stable Diffusion 1.x, 2.x, XL and 3 variants, along with ControlNet for non-XL models. It also includes weight compression options such as palettization down to 6 bits, mixed-bit palettization for lower sizes, and activation quantization for newer chips. Pre-converted models are published on the Hugging Face Hub.
The project lists target requirements of macOS 13.1 or later for conversion and iOS 16.2 or later for runtime, with M1-generation or newer Mac and iPad hardware and A14 or newer iPhones. Published benchmarks cover several iPhone, iPad and Mac models.
Key features
- 01Converts Stable Diffusion PyTorch checkpoints to Core ML .mlpackage files.
- 02Provides a Python CLI and pipeline built on Hugging Face diffusers.
- 03Ships a Swift package for integrating image generation into Apple apps.
- 04Supports SD 1.x, 2.x, SDXL, and SD 3 model variants, plus ControlNet for non-XL models.
- 05Offers weight compression from 6-bit palettization down to mixed-bit palettization.
Use cases
- Running Stable Diffusion locally on a Mac or iPad without a cloud GPU.
- Adding on-device image generation to an iOS or macOS app built with Swift.
- Converting a fine-tuned Stable Diffusion checkpoint into Core ML for Apple devices.
- Experimenting with model compression to reduce memory use on iPhones.
Things to know
- Targets Apple Silicon hardware only, requiring M-series Macs or iPad/iPhone devices.
- Some memory improvements need macOS 14 or iOS 17, and conversion pins Python 3.8 and coremltools 7.0.
Topics
FAQ
- What is ml-stable-diffusion?
- Converts Stable Diffusion PyTorch models to Core ML and runs image generation on Apple Silicon devices via a Python package and a Swift package. ml-stable-diffusion is Apple's toolkit for running Stable Diffusion on Apple Silicon using Core ML. It has two parts: a Python package that converts PyTorch checkpoints from Hugging Face into Core ML model files and generates images with the diffusers library, and a Swift package that can be added to Xcode projects to run those Core ML models inside macOS, iOS or iPadOS apps.
- Is ml-stable-diffusion free to use? What license does it have?
- ml-stable-diffusion is open source under the MIT license. Check the license terms before using it in commercial products.
- Is ml-stable-diffusion still maintained?
- Updated within 6 months. The last commit to ml-stable-diffusion was on Sep 11, 2026.
- What can I use ml-stable-diffusion for?
- Running Stable Diffusion locally on a Mac or iPad without a cloud GPU. Adding on-device image generation to an iOS or macOS app built with Swift. Converting a fine-tuned Stable Diffusion checkpoint into Core ML for Apple devices. Experimenting with model compression to reduce memory use on iPhones.
- Where can I get ml-stable-diffusion?
- The source code and installation instructions are on GitHub: https://github.com/apple-aiml-research/ml-stable-diffusion
Summary generated by AI from the project's README. GitHub stats refresh daily.
Related projects
Open-source Stable Diffusion-based tool for generating character-consistent comic images and long-range image-to-video sequences.
Native Apple MLX implementations of recent image generation models (FLUX, Z-Image, Qwen Image, and more) for running locally on a Mac.
A native macOS app for generating images locally with Stable Diffusion and FLUX.2 Klein on Apple Silicon.
Swift SDK for running speech-to-text, text-to-speech and speaker diarization fully on-device on Apple silicon.
Swift and Metal runtime that runs the 26B Gemma 4 26B-A4B model on Apple Silicon Macs with about 2 GB of RAM by streaming experts from SSD.