Repolode
Submit
simplexai-labs

simplexai-labs/PolyStrata

Runs large open-weight LLMs such as Qwen3.6, GLM 5.3 Flash and Gemma 4 on your own NVIDIA or AMD PC, exposing OpenAI- and Anthropic-compatible APIs on localhost.

View on GitHub
Stars
906
Forks
128
Open issues
15
Last commit
3 days ago

What is it?

PolyStrata is a local inference setup for large GGUF language models on consumer hardware. It runs twelve models across two engines: Qwen3.8-Flash-Next on the Strata engine for NVIDIA or AMD cards with 12 GB or more, and eleven other models (including GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Gemma 4 and MiniCPM-V) on a PolyStrata-specific engine. The model files are large and most of their weights stay in system RAM, with the GPU holding the frequently used parts.

The project ships a one-click installer for Windows (START-HERE.bat) and Linux (setup.sh), plus a Mac build for Qwen3.8-Flash-Next on Apple Silicon. The installer detects the graphics card and RAM, recommends a model size and context length, downloads the model and starts a local server at 127.0.0.1:8080. The server exposes an OpenAI-compatible endpoint at /v1 and an Anthropic-style /v1/messages endpoint, and includes a browser chat and a live monitor.

The documentation lists benchmark figures per model and hardware, including tokens per second for answers and prompt reading, and notes that the PolyStrata-engine models are experimental.

Key features

  • 01Runs twelve large open-weight models, including GLM 5.3 Flash, DeepSeek V4 Flash and Qwen3.6, on local hardware.
  • 02One-click installers for Windows and Linux that pick a model size for your GPU and RAM.
  • 03Exposes OpenAI-compatible and Anthropic-style APIs on localhost for apps and coding agents.
  • 04Includes a browser chat interface and a live monitor of GPU, CPU and RAM usage.
  • 05Supports multi-GPU setups and optional image input on several models.

Use cases

  • Running a capable coding assistant such as Claude Code against a model that stays on your own machine.
  • Chatting with large models privately when cloud APIs are not an option.
  • Testing large MoE models on a single gaming-class GPU with plenty of system RAM.
  • Pointing existing OpenAI- or Anthropic-based apps at a local endpoint without code changes.

Quick start

  1. 1

    Download PolyStrata from GitHub as a zip or clone it

  2. 2

    On Windows, double-click START-HERE.bat in the PolyStrata folder

  3. 3

    On Linux, run the setup script in the PolyStrata folder

    ./setup.sh
  4. 4

    Wait for the model to download and start, then open http://127.0.0.1:8080

Things to know

  • Most models need an NVIDIA or AMD graphics card with at least 8 to 12 GB of VRAM.
  • Eleven of the twelve models run on an experimental engine, and GLM 5.3 Flash and DeepSeek V4 Flash need a 32 GB card and Linux.
  • Model downloads are large, from 11 GB up to 114 GB, and the first start can freeze the PC for 1 to 3 minutes.

In collections

Self-hosted appsRun LLMs locally

Topics

#local-llm#inference-engine#llm#gguf#openai-api#anthropic-api#cuda#self-hosted#gemma#glm

FAQ

What is PolyStrata?
Runs large open-weight LLMs such as Qwen3.6, GLM 5.3 Flash and Gemma 4 on your own NVIDIA or AMD PC, exposing OpenAI- and Anthropic-compatible APIs on localhost. PolyStrata is a local inference setup for large GGUF language models on consumer hardware. It runs twelve models across two engines: Qwen3.8-Flash-Next on the Strata engine for NVIDIA or AMD cards with 12 GB or more, and eleven other models (including GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Gemma 4 and MiniCPM-V) on a PolyStrata-specific engine. The model files are large and most of their weights stay in system RAM, with the GPU holding the frequently used parts.
Is PolyStrata free to use? What license does it have?
PolyStrata is open source under the MIT license. Check the license terms before using it in commercial products.
Is PolyStrata still maintained?
Actively maintained. The last commit to PolyStrata was on Nov 5, 2026.
What can I use PolyStrata for?
Running a capable coding assistant such as Claude Code against a model that stays on your own machine. Chatting with large models privately when cloud APIs are not an option. Testing large MoE models on a single gaming-class GPU with plenty of system RAM. Pointing existing OpenAI- or Anthropic-based apps at a local endpoint without code changes.
How do I install PolyStrata?
Download PolyStrata from GitHub as a zip or clone it On Windows, double-click START-HERE.bat in the PolyStrata folder On Linux, run the setup script in the PolyStrata folder: ./setup.sh Wait for the model to download and start, then open http://127.0.0.1:8080
Where can I get PolyStrata?
The source code and installation instructions are on GitHub: https://github.com/simplexai-labs/PolyStrata

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects