Efficient, Flexible and Portable Structured Generation

Get Started | Documentation | Blogpost | Technical Report

Approaching-AI K3 backports

This branch contains our XGrammar v0.2.1-based maxLength, minLength, and FSM 32-bit index fixes. It is an internal build, not the upstream PyPI release. See the patch history, validation, and installation notes before building or upgrading. The upstream README continues below.

News

• [2026/5] XGrammar-2 has been released! Check out our blog for more information.
• [2025/12] XGrammar has been officially integrated into Mirai
• [2025/09] XGrammar has been officially integrated into OpenVINO GenAI
• [2025/02] XGrammar has been officially integrated into Modular's MAX
• [2025/01] XGrammar has been officially integrated into TensorRT-LLM.
• [2024/12] XGrammar has been officially integrated into vLLM.
• [2024/12] We presented research talks on XGrammar at CMU, UC Berkeley, MIT, THU, SJTU, Ant Group, LMSys, Qingke AI, Camel AI. The slides can be found here.
• [2024/11] XGrammar has been officially integrated into SGLang.
• [2024/11] XGrammar has been officially integrated into MLC-LLM.
• [2024/11] We officially released XGrammar v0.1.0!

Overview

XGrammar is an open-source library for efficient, flexible, and portable structured generation.

It leverages constrained decoding to ensure 100% structural correctness of the output. It supports general context-free grammar to enable a broad range of structures, including JSON, regex, custom context-free grammar, etc.

XGrammar uses careful optimizations to achieve extremely low overhead in structured generation. It has achieved near-zero overhead in JSON generation, making it one of the fastest structured generation engines available.

XGrammar features universal deployment. It supports:

• Platforms: Linux, macOS, Windows
• Hardware: CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU, etc.
• Languages: Python, C++, and JavaScript APIs
• Models: Qwen, Llama, DeepSeek, Phi, Gemma, etc.

XGrammar is very easy to integrate with LLM inference engines. It is the default structured generation backend for most LLM inference engines, including vLLM, SGLang, TensorRT-LLM, and MLC-LLM, as well as many other companies. You can also try out their structured generation modes!

Get Started

Install XGrammar:

pip install xgrammar

For use with MPS on Apple Silicon, install with:

pip install "xgrammar[metal]"

Import XGrammar:

import xgrammar as xgr

Please visit our documentation to get started with XGrammar.

• Installation
• Quick start

Third-Party Bindings

• Rust: xgrammar-rs — Community Rust bindings for XGrammar.

Collaborators

XGrammar has been widely adopted in industry, open-source projects, and academia. Our collaborators include:

                      WebLLM  

Citation

If you find XGrammar useful in your research, please consider citing our papers:

@article{dong2024xgrammar,
  title={Xgrammar: Flexible and efficient structured generation engine for large language models},
  author={Dong, Yixin and Ruan, Charlie F and Cai, Yaxing and Lai, Ruihang and Xu, Ziyi and Zhao, Yilong and Chen, Tianqi},
  journal={Proceedings of Machine Learning and Systems 7},
  year={2024}
}
@misc{li2026xgrammar2efficientdynamicstructured,
  title={XGrammar-2: Efficient Dynamic Structured Generation Engine for Agentic LLMs},
  author={Linzhang Li and Yixin Dong and Guanjie Wang and Ziyi Xu and Alexander Jiang and Tianqi Chen},
  year={2026},
  eprint={2601.04426},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2601.04426},
}