Implementation:InternLM Lmdeploy XgrammarBind

Knowledge Sources	InternLM_Lmdeploy
Domains	Python Bindings, Structured Output
Last Updated	2026-02-07 15:00 GMT

Overview

Provides pybind11 Python bindings for the xgrammar library, exposing TokenizerInfo, CompiledGrammar, and GrammarCompiler classes for grammar-guided decoding configuration from Python.

Description

This file creates the _xgrammar Python extension module using pybind11, enabling Python-level configuration of grammar-guided decoding for TurboMind. It is adapted from the xgrammar project's original nanobind bindings.

The module exposes three classes:

TokenizerInfo: Represents tokenizer metadata needed by the grammar compiler. The Python binding supports construction from an encoded vocabulary (list of str or bytes), a vocabulary type integer (0, 1, or 2 mapped to VocabType enum), optional vocab size, optional stop token IDs, and an add_prefix_space flag. Read-only properties expose vocab_type, vocab_size, add_prefix_space, decoded_vocab (as bytes), stop_token_ids, and special_token_ids. Methods include dump_metadata() for serialization, from_vocab_and_metadata() for deserialization, and _detect_metadata_from_hf() for auto-detection from Hugging Face tokenizers.

Helper functions handle type conversion: CommonEncodedVocabType() converts a Python list of mixed str/bytes to a C++ vector of strings, and TokenizerInfo_GetDecodedVocab() converts decoded vocab entries to Python bytes objects.

CompiledGrammar: A thin wrapper class (no additional methods exposed) representing a compiled grammar ready for use in guided decoding.

GrammarCompiler: The main compilation entry point. Constructed with a TokenizerInfo, optional max threads (default 8), cache enable flag (default true), and max memory bytes (default -1 for unlimited). Provides compile_json_schema() (with options for whitespace handling, indentation, separators, strict mode) and compile_regex() methods. Both methods release the GIL during compilation for better concurrency.

Usage

Imported in Python as _xgrammar (typically wrapped by a higher-level Python module). Used to create TokenizerInfo from the model's tokenizer, instantiate a GrammarCompiler, and compile JSON schemas or regex patterns into CompiledGrammar objects that are then passed to ModelRequest via setGrammar().

Code Reference

Source Location

• Repository: InternLM_Lmdeploy
• File: src/turbomind/python/xgrammar_bind.cpp
• Lines: 1-134

Signature

PYBIND11_MODULE(_xgrammar, m)
{
    py::class_<TokenizerInfo, std::shared_ptr<TokenizerInfo>>(m, "TokenizerInfo")
        .def(py::init<...>())
        .def_property_readonly("vocab_type", ...)
        .def_property_readonly("vocab_size", ...)
        .def_property_readonly("add_prefix_space", ...)
        .def_property_readonly("decoded_vocab", ...)
        .def_property_readonly("stop_token_ids", ...)
        .def_property_readonly("special_token_ids", ...)
        .def("dump_metadata", ...)
        .def_static("from_vocab_and_metadata", ...)
        .def_static("_detect_metadata_from_hf", ...);

    py::class_<CompiledGrammar>(m, "CompiledGrammar");

    py::class_<GrammarCompiler>(m, "GrammarCompiler")
        .def(py::init<const TokenizerInfo&, int, bool, int64_t>())
        .def("compile_json_schema", ...)
        .def("compile_regex", ...);
}

Import

import _xgrammar

I/O Contract

Inputs

Name	Type	Required	Description
encoded_vocab	List[str or bytes]	Yes	Tokenizer vocabulary entries
vocab_type	int	Yes	Vocabulary type: 0, 1, or 2
vocab_size	Optional[int]	No	Override vocabulary size
stop_token_ids	Optional[List[int]]	No	Token IDs that signal generation stop
add_prefix_space	bool	Yes	Whether the tokenizer adds prefix space
schema (compile_json_schema)	str	Yes	JSON schema string to compile
schema (compile_regex)	str	Yes	Regex pattern string to compile

Outputs

Name	Type	Description
TokenizerInfo	object	Tokenizer metadata for grammar compilation
CompiledGrammar	object	Compiled grammar ready for guided decoding

Usage Examples

import _xgrammar

# Create tokenizer info
tokenizer_info = _xgrammar.TokenizerInfo(
    encoded_vocab=vocab_list,
    vocab_type=0,
    vocab_size=32000,
    stop_token_ids=[2],
    add_prefix_space=False
)

# Create compiler
compiler = _xgrammar.GrammarCompiler(tokenizer_info, max_threads=8)

# Compile a JSON schema
grammar = compiler.compile_json_schema('{"type": "object", "properties": {"name": {"type": "string"}}}')

# Compile a regex pattern
grammar = compiler.compile_regex(r'\d{3}-\d{4}')

Related Pages

• Environment:InternLM_Lmdeploy_CUDA_GPU_Runtime