A launcher and patched llama.cpp setup that runs the 1.58-bit Ternary Bonsai 2 27B model with a 262k context on 12 GB and 8 GB NVIDIA GPUs.
796PythonOtherUpdated 3 hours ago
#llama-cpp
Open-source projects tagged “llama-cpp”
Curated GitHub repositories related to llama-cpp, with AI summaries and key stats.
Sort:Recently added
2 repositoriesClear filters
Serving setup for Mirai S 27B (a Qwen3.8-27B GGUF) on a 12 GB RTX 4070, with a full 262k-token window and q8_0 K/V cache.
10110PythonOtherUpdated 5 hours ago