Implementation:Ggml org Llama cpp Grammar
| Knowledge Sources | |
|---|---|
| Domains | Grammar, Constrained_Generation |
| Last Updated | 2026-02-15 00:00 GMT |
Overview
Implements the grammar-constrained generation system for llama.cpp, including GBNF grammar parsing, grammar state management, and token filtering based on grammar rules.
Description
This file provides the core grammar engine that enables structured output generation (JSON, code, etc.) by constraining token sampling to only produce outputs conforming to a specified grammar. It includes UTF-8 decoding helpers, a GBNF grammar parser (`llama_grammar_parser`) that converts grammar text into rule vectors with character ranges, alternates, and rule references, and a pushdown-stack-based state machine that tracks valid parse positions. The system supports lazy grammars with trigger tokens/patterns for deferred activation, and both token-level and character-level matching.
Usage
Use this module when you need to constrain LLM output to follow a specific grammar format, such as generating valid JSON, code in a particular language, or any structured text format defined by a GBNF grammar specification.
Code Reference
Source Location
- Repository: Ggml_org_Llama_cpp
- File: src/llama-grammar.cpp
- Lines: 1-1464
Signature
// Grammar parser
uint32_t llama_grammar_parser::get_symbol_id(const char * src, size_t len);
uint32_t llama_grammar_parser::generate_symbol_id(const std::string & base_name);
void llama_grammar_parser::add_rule(uint32_t rule_id, const llama_grammar_rule & rule);
bool llama_grammar_parser::parse(const char * src);
void llama_grammar_parser::print(FILE * file);
llama_grammar_stack llama_grammar_parser::c_rules() const;
// Grammar state operations
static void llama_grammar_advance_stack(/* ... */);
static llama_grammar_candidates llama_grammar_reject_candidates(/* ... */);
static bool llama_grammar_detect_left_recursion(/* ... */);
llama_grammar_stacks & llama_grammar_get_stacks(struct llama_grammar * grammar);
// Trigger pattern matching
size_t llama_grammar_trigger_pattern::find(const std::string & input) const;
Import
#include "llama-grammar.h"
#include "llama-impl.h"
#include "llama-vocab.h"
#include "llama-sampler.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| src | const char * | Yes | GBNF grammar text to parse into rule vectors |
| grammar | llama_grammar * | Yes | Grammar state object containing stacks and rules |
| candidates | llama_grammar_candidates | Yes | Token candidates to filter against grammar constraints |
| input | const std::string & | No | Input string for trigger pattern matching in lazy grammars |
Outputs
| Name | Type | Description |
|---|---|---|
| parse result | bool | Whether the GBNF grammar was parsed successfully |
| c_rules | llama_grammar_stack | Compiled grammar rules as a vector of rule vectors |
| rejected candidates | llama_grammar_candidates | Tokens that do not conform to the current grammar state |
| stacks | llama_grammar_stacks & | Reference to the current grammar parse state stacks |
Usage Examples
// Parse a GBNF grammar
llama_grammar_parser parser;
bool ok = parser.parse(gbnf_text);
llama_grammar_stack rules = parser.c_rules();
// Get grammar stacks for constrained sampling
llama_grammar_stacks & stacks = llama_grammar_get_stacks(grammar);
// Reject candidates that don't conform to grammar
llama_grammar_candidates rejected = llama_grammar_reject_candidates(rules, stacks, candidates);