Open-source projects tagged “speculative-decoding”

Pairs an iPhone with a Mac over USB-C so the phone's GPU helps run a Qwen 27B model locally, speeding up prompt reading and extending context.

Open source inference engine that tunes its kernels to your hardware to run open models faster on Apple Silicon, NVIDIA, AMD, or CPU.