Repolode
Submit
mlc-ai

mlc-ai/xgrammar

A fast, flexible library for structured LLM generation that constrains model output to JSON, regex, or custom context-free grammars.

View on GitHubWebsite
Stars
1,967
Forks
241
Open issues
130
Last commit
3 days ago

What is it?

XGrammar is an open-source library for structured generation with large language models. It uses constrained decoding so that the generated output follows a specified structure, such as JSON, a regular expression, or a general context-free grammar. This addresses the common problem of LLM output failing to parse or match an expected schema.

The project focuses on keeping the overhead of constrained decoding low, and it reports near-zero overhead for JSON generation. It is designed for portability across Linux, macOS and Windows, and across CPU, NVIDIA and AMD GPUs, Apple Silicon and TPUs. Bindings are provided for Python, C++, JavaScript and Swift.

XGrammar is intended to be embedded in inference engines. It is integrated into projects such as vLLM, SGLang, TensorRT-LLM and MLC-LLM, and it can be installed with pip.

Key features

  • 01Constrains output to JSON, regex, or custom context-free grammars.
  • 02Targets near-zero overhead for JSON generation.
  • 03Runs on Linux, macOS and Windows across CPU, NVIDIA, AMD, Apple Silicon and TPU.
  • 04Provides Python, C++, JavaScript and Swift APIs.
  • 05Installable with pip, with an optional Metal extra for Apple Silicon.

Use cases

  • Forcing an LLM serving engine to return valid JSON for downstream parsing.
  • Constraining model output to a regex or grammar for tool calls or function arguments.
  • Adding structured output support to a custom LLM inference stack.

Quick start

  1. 1

    Install XGrammar from PyPI

    pip install xgrammar
  2. 2

    Install with Metal support for Apple Silicon (optional)

    pip install "xgrammar[metal]"
  3. 3

    Import XGrammar in your Python code

    import xgrammar as xgr

Things to know

  • Most of the README's usage details live in external documentation, not in the README itself.

Topics

#large-language-models#structured-generation#constrained-decoding#json#llm-inference#context-free-grammar

FAQ

What is xgrammar?
A fast, flexible library for structured LLM generation that constrains model output to JSON, regex, or custom context-free grammars. XGrammar is an open-source library for structured generation with large language models. It uses constrained decoding so that the generated output follows a specified structure, such as JSON, a regular expression, or a general context-free grammar. This addresses the common problem of LLM output failing to parse or match an expected schema.
Is xgrammar free to use? What license does it have?
xgrammar is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is xgrammar still maintained?
Actively maintained. The last commit to xgrammar was on Oct 18, 2026.
What can I use xgrammar for?
Forcing an LLM serving engine to return valid JSON for downstream parsing. Constraining model output to a regex or grammar for tool calls or function arguments. Adding structured output support to a custom LLM inference stack.
How do I install xgrammar?
Install XGrammar from PyPI: pip install xgrammar Install with Metal support for Apple Silicon (optional): pip install "xgrammar[metal]" Import XGrammar in your Python code: import xgrammar as xgr
Where can I get xgrammar?
The source code and installation instructions are on GitHub: https://github.com/mlc-ai/xgrammar

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects