Implementation:Ggml org Llama cpp Chat Peg Parser Header

Knowledge Sources	Ggml_org_Llama_cpp
Domains	Chat, Parsing
Last Updated	2026-02-15 00:00 GMT

Overview

Declares the PEG parser builder and mapper class hierarchies for constructing PEG grammars and converting their AST output into structured chat messages.

Description

Defines three builder hierarchies that extend `common_peg_parser_builder`: `common_chat_peg_builder` (base, with REASONING/CONTENT tags), `common_chat_peg_native_builder` (adds TOOL_OPEN/NAME/ID/ARGS tags for models with native JSON tool arguments), and `common_chat_peg_constructed_builder` (adds TOOL_ARG_NAME/STRING_VALUE/JSON_VALUE tags for models with structured per-argument output). Each builder has a corresponding mapper class that traverses PEG AST nodes and populates a `common_chat_msg`. Inline helper functions (`build_chat_peg_parser`, `build_chat_peg_native_parser`, `build_chat_peg_constructed_parser`) simplify parser construction via lambdas.

Usage

Use this header when implementing PEG-based parsing for chat model output formats. The three builder levels support different tool call structure granularities, from simple content/reasoning extraction to full per-argument tool call parsing.

Code Reference

Source Location

• Repository: Ggml_org_Llama_cpp
• File: common/chat-peg-parser.h
• Lines: 1-105

Signature

// Base builder - reasoning and content tags
class common_chat_peg_builder : public common_peg_parser_builder {
  public:
    static constexpr const char * REASONING_BLOCK = "reasoning-block";
    static constexpr const char * REASONING = "reasoning";
    static constexpr const char * CONTENT = "content";
    common_peg_parser reasoning_block(const common_peg_parser & p);
    common_peg_parser reasoning(const common_peg_parser & p);
    common_peg_parser content(const common_peg_parser & p);
};

// Native builder - tool calls with JSON args
class common_chat_peg_native_builder : public common_chat_peg_builder {
  public:
    static constexpr const char * TOOL_OPEN = "tool-open";
    static constexpr const char * TOOL_NAME = "tool-name";
    static constexpr const char * TOOL_ID = "tool-id";
    static constexpr const char * TOOL_ARGS = "tool-args";
    // ... builder methods
};

// Constructed builder - per-argument tool calls
class common_chat_peg_constructed_builder : public common_chat_peg_builder {
  public:
    static constexpr const char * TOOL_ARG_NAME = "tool-arg-name";
    static constexpr const char * TOOL_ARG_STRING_VALUE = "tool-arg-string-value";
    static constexpr const char * TOOL_ARG_JSON_VALUE = "tool-arg-json-value";
    // ... builder methods
};

// Mapper classes
class common_chat_peg_mapper { /* base mapper */ };
class common_chat_peg_native_mapper : public common_chat_peg_mapper { /* native tool call mapper */ };
class common_chat_peg_constructed_mapper : public common_chat_peg_mapper { /* constructed tool call mapper */ };

// Factory functions
inline common_peg_arena build_chat_peg_parser(const std::function<...> & fn);
inline common_peg_arena build_chat_peg_native_parser(const std::function<...> & fn);
inline common_peg_arena build_chat_peg_constructed_parser(const std::function<...> & fn);

Import

#include "chat.h"
#include "peg-parser.h"

I/O Contract

Inputs

Name	Type	Required	Description
fn	std::function lambda	Yes	Lambda receiving a builder reference, returning a PEG parser root rule
msg	common_chat_msg&	Yes	Mutable chat message reference to populate during AST mapping
node	common_peg_ast_node	Yes	Individual AST node to map

Outputs

Name	Type	Description
common_peg_arena	object	Compiled PEG grammar arena ready for parsing
common_chat_msg	struct	Populated chat message with content, reasoning, and tool calls

Usage Examples

#include "chat-peg-parser.h"

// Build a native PEG parser for tool calls
auto arena = build_chat_peg_native_parser([](common_chat_peg_native_builder & b) {
    auto tool = b.tool(b.seq({
        b.tool_open(b.lit("<tool_call>")),
        b.tool_name(b.until(b.lit("\n"))),
        b.tool_args(b.until(b.lit("</tool_call>"))),
        b.tool_close(b.lit("</tool_call>"))
    }));
    return b.seq({
        b.opt(b.reasoning_block(b.reasoning(b.until(b.lit("</think>"))))),
        b.content(b.until(b.lookahead(b.lit("<tool_call>")))),
        b.many(tool)
    });
});

// Parse and map
auto result = arena.parse(model_output);
common_chat_msg msg;
common_chat_peg_native_mapper mapper(msg);
mapper.from_ast(arena.ast_arena(), result);

Related Pages

• Principle:Ggml_org_Llama_cpp_Chat_Parsing