Repolode
Submit
StayLameBro

StayLameBro/backburner

Pairs an iPhone with a Mac over USB-C so the phone's GPU helps run a Qwen 27B model locally, speeding up prompt reading and extending context.

View on GitHub
Stars
1,072
Forks
120
Open issues
8
Last commit
7 days ago

What is it?

Backburner is a local LLM inference server for Apple Silicon Macs that uses a connected iPhone as a second compute device. Over a 10 Gb/s USB-C cable, the Mac runs the first 40 of 64 layers of a Qwen 27B model while the iPhone runs the remaining layers on its GPU, pipelined so both work at once. This targets prompt prefill, which matters when an agent reads long files or tool outputs.

The iPhone can also hold the oldest part of the context and compute attention over it, letting the server handle much longer contexts than a 24 GB Mac alone. The server exposes an OpenAI-compatible endpoint on localhost. The engine is a llama.cpp fork with Mac-specific kernels (SME2, Metal, speculative decoding), and the project's README reports identical greedy output with and without the phone in its tests.

The project is pre-release and tested on an M4 Pro Mac with iPhone 17 Pro Max and 16 Pro Max. Setup uses an installer script, a sideloaded iPhone app that works with a free Apple ID, and separate model downloads.

Key features

  • 01Splits layers across a Mac GPU and an iPhone GPU for faster prompt prefill.
  • 02Extends usable context by moving the oldest attention keys to the iPhone.
  • 03Provides an OpenAI-compatible local API server.
  • 04Uses a saved-prompt cache on disk to skip re-reading known system prompts.
  • 05Runs over a USB-C cable with a sideloaded iPhone app, no jailbreak needed.

Use cases

  • Running a 27B local coding agent on a 24 GB MacBook that reads large files or tool outputs.
  • Experimenting with long-context local inference on Apple Silicon hardware you already own.
  • Testing split-device inference across Mac and iPhone.

Quick start

  1. 1

    Get an Apple Silicon Mac and an iPhone 15 Pro or newer, connected by a 10 Gb/s USB-C cable

  2. 2

    Run the installer, setting TAIL_L to your device's split if you want a value other than the default

Things to know

  • Requires an Apple Silicon Mac and a recent iPhone Pro model, so it does not run on other hardware yet.
  • Past 64k context, the iPhone app must stay open and in front, or the server stops.
  • Only one request at a time is supported (-np 1).
  • Benchmarks come mainly from the maintainer's own M4 Pro Mac and iPhone 17 Pro Max.

Topics

#local-llm#llm-inference#apple-silicon#metal#llama-cpp#ios#speculative-decoding#iphone#macos#qwen

FAQ

What is backburner?
Pairs an iPhone with a Mac over USB-C so the phone's GPU helps run a Qwen 27B model locally, speeding up prompt reading and extending context. Backburner is a local LLM inference server for Apple Silicon Macs that uses a connected iPhone as a second compute device. Over a 10 Gb/s USB-C cable, the Mac runs the first 40 of 64 layers of a Qwen 27B model while the iPhone runs the remaining layers on its GPU, pipelined so both work at once. This targets prompt prefill, which matters when an agent reads long files or tool outputs.
Is backburner free to use? What license does it have?
backburner is open source under the MIT license. Check the license terms before using it in commercial products.
Is backburner still maintained?
Actively maintained. The last commit to backburner was on Oct 13, 2026.
What can I use backburner for?
Running a 27B local coding agent on a 24 GB MacBook that reads large files or tool outputs. Experimenting with long-context local inference on Apple Silicon hardware you already own. Testing split-device inference across Mac and iPhone.
How do I install backburner?
Get an Apple Silicon Mac and an iPhone 15 Pro or newer, connected by a 10 Gb/s USB-C cable Run the installer, setting TAIL_L to your device's split if you want a value other than the default
Where can I get backburner?
The source code and installation instructions are on GitHub: https://github.com/StayLameBro/backburner

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects