Repolode
Submit
doofzoff

doofzoff/SIMURG

A numpy-only Python guard that detects LLM decoding corruption such as repetition loops and language drift mid-stream and aborts before users see bad tokens.

View on GitHub
Stars
101
Forks
15
Open issues
0
Last commit
29 days ago

What is it?

SIMURG watches a token stream from an LLM as it is generated and flags decoding corruption: repetition collapse, cross-lingual drift, regurgitation of boilerplate, structural garbage, and leaked chat-template markers. It holds the first 350 characters before releasing them, re-checks the stream every 400 characters, and cuts it when a calibrated threshold is crossed, so corrupt text is not shown to the user. Each alarm comes with human-readable reasons and an estimated onset position.

The core runs on numpy only, with no model or GPU required, and works with any OpenAI-compatible chat completions endpoint such as vLLM, llama.cpp server, TGI, Ollama, or OpenAI. A GuardedLLM wrapper adds retry and fallback-model logic, and a self-heal path can continue from the verified clean prefix instead of regenerating the whole answer. Thresholds are calibrated on clean outputs, and the project includes a retraining command for your own traffic and a dashboard for live guarding and session replay.

The project is explicit that it detects delivery corruption, not factual errors: a fluent but false sentence has no statistical signature. It also includes an optional small transformer tier, a web-search layer for grounding checks, and an experimental feedback-learning module.

Key features

  • 01Detects repetition loops, language drift, regurgitation, structural garbage and template leakage in streamed output.
  • 02Holds the opening characters and cuts the stream on a calibrated threshold, so corrupt text is not shown.
  • 03Runs on numpy alone with no model or GPU, and works with any OpenAI-compatible endpoint.
  • 04Reports human-readable alarm reasons and estimated onset position.
  • 05Retries or falls back to another model when an attempt is flagged.
  • 06Supports retraining thresholds on your own clean outputs and a live guard dashboard.

Use cases

  • Protecting user-facing chatbots built on quantized or self-hosted models from degenerate output.
  • Adding a mid-stream quality gate in front of an inference server before tokens reach the UI.
  • Auditing and replaying corrupt generations from a production model to find failure patterns.
  • Detecting a new failure mode in your own model by training a custom detector from labeled examples.

Quick start

  1. 1

    Install the package from PyPI

    pip install simurg
  2. 2

    Or install from source by cloning the repository

    git clone https://github.com/doofzoff/SIMURG.git
  3. 3

    Enter the project folder and install it in editable mode

    cd SIMURG && pip install -e .
  4. 4

    Run the reproducible benchmark

    python3 -m simurg.data.evaluate

Things to know

  • Its calibration is tuned to the bundled synthetic benchmark, so you should retrain on your own traffic before relying on it in production.
  • Mid-stream corruption may leave a bad tail of up to about 900 characters before the retry replaces it, and corruption starting in the hold window is only blocked fully in 12 of 21 benchmark cases.

Topics

#llm#guardrails#streaming#llm-safety#hallucination-detection#anomaly-detection#openai-api#vllm#inference#online-learning

FAQ

What is SIMURG?
A numpy-only Python guard that detects LLM decoding corruption such as repetition loops and language drift mid-stream and aborts before users see bad tokens. SIMURG watches a token stream from an LLM as it is generated and flags decoding corruption: repetition collapse, cross-lingual drift, regurgitation of boilerplate, structural garbage, and leaked chat-template markers. It holds the first 350 characters before releasing them, re-checks the stream every 400 characters, and cuts it when a calibrated threshold is crossed, so corrupt text is not shown to the user. Each alarm comes with human-readable reasons and an estimated onset position.
Is SIMURG free to use? What license does it have?
SIMURG is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is SIMURG still maintained?
Actively maintained. The last commit to SIMURG was on Sep 21, 2026.
What can I use SIMURG for?
Protecting user-facing chatbots built on quantized or self-hosted models from degenerate output. Adding a mid-stream quality gate in front of an inference server before tokens reach the UI. Auditing and replaying corrupt generations from a production model to find failure patterns. Detecting a new failure mode in your own model by training a custom detector from labeled examples.
How do I install SIMURG?
Install the package from PyPI: pip install simurg Or install from source by cloning the repository: git clone https://github.com/doofzoff/SIMURG.git Enter the project folder and install it in editable mode: cd SIMURG && pip install -e . Run the reproducible benchmark: python3 -m simurg.data.evaluate
Where can I get SIMURG?
The source code and installation instructions are on GitHub: https://github.com/doofzoff/SIMURG

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects