A launcher and patched llama.cpp setup that runs the 1.58-bit Ternary Bonsai 2 27B model with a 262k context on 12 GB and 8 GB NVIDIA GPUs.
796PythonOtherUpdated 5 hours ago
#speculative-decoding
Open-source projects tagged “speculative-decoding”
Curated GitHub repositories related to speculative-decoding, with AI summaries and key stats.