Repolode
apple-aiml-research

apple-aiml-research/ml-stable-diffusion

Converts Stable Diffusion PyTorch models to Core ML and runs image generation on Apple Silicon devices via a Python package and a Swift package.

Stars
17,968
Forks
1,080
Open issues
192
Last commit
last month

What is it?

ml-stable-diffusion is Apple's toolkit for running Stable Diffusion on Apple Silicon using Core ML. It has two parts: a Python package that converts PyTorch checkpoints from Hugging Face into Core ML model files and generates images with the diffusers library, and a Swift package that can be added to Xcode projects to run those Core ML models inside macOS, iOS or iPadOS apps.

The conversion step supports Stable Diffusion 1.x, 2.x, XL and 3 variants, along with ControlNet for non-XL models. It also includes weight compression options such as palettization down to 6 bits, mixed-bit palettization for lower sizes, and activation quantization for newer chips. Pre-converted models are published on the Hugging Face Hub.

The project lists target requirements of macOS 13.1 or later for conversion and iOS 16.2 or later for runtime, with M1-generation or newer Mac and iPad hardware and A14 or newer iPhones. Published benchmarks cover several iPhone, iPad and Mac models.

Key features

  • 01Converts Stable Diffusion PyTorch checkpoints to Core ML .mlpackage files.
  • 02Provides a Python CLI and pipeline built on Hugging Face diffusers.
  • 03Ships a Swift package for integrating image generation into Apple apps.
  • 04Supports SD 1.x, 2.x, SDXL, and SD 3 model variants, plus ControlNet for non-XL models.
  • 05Offers weight compression from 6-bit palettization down to mixed-bit palettization.

Use cases

  • Running Stable Diffusion locally on a Mac or iPad without a cloud GPU.
  • Adding on-device image generation to an iOS or macOS app built with Swift.
  • Converting a fine-tuned Stable Diffusion checkpoint into Core ML for Apple devices.
  • Experimenting with model compression to reduce memory use on iPhones.

Things to know

  • Targets Apple Silicon hardware only, requiring M-series Macs or iPad/iPhone devices.
  • Some memory improvements need macOS 14 or iOS 17, and conversion pins Python 3.8 and coremltools 7.0.

Topics

FAQ

What is ml-stable-diffusion?
Converts Stable Diffusion PyTorch models to Core ML and runs image generation on Apple Silicon devices via a Python package and a Swift package. ml-stable-diffusion is Apple's toolkit for running Stable Diffusion on Apple Silicon using Core ML. It has two parts: a Python package that converts PyTorch checkpoints from Hugging Face into Core ML model files and generates images with the diffusers library, and a Swift package that can be added to Xcode projects to run those Core ML models inside macOS, iOS or iPadOS apps.
Is ml-stable-diffusion free to use? What license does it have?
ml-stable-diffusion is open source under the MIT license. Check the license terms before using it in commercial products.
Is ml-stable-diffusion still maintained?
Updated within 6 months. The last commit to ml-stable-diffusion was on Sep 11, 2026.
What can I use ml-stable-diffusion for?
Running Stable Diffusion locally on a Mac or iPad without a cloud GPU. Adding on-device image generation to an iOS or macOS app built with Swift. Converting a fine-tuned Stable Diffusion checkpoint into Core ML for Apple devices. Experimenting with model compression to reduce memory use on iPhones.
Where can I get ml-stable-diffusion?
The source code and installation instructions are on GitHub: https://github.com/apple-aiml-research/ml-stable-diffusion

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects