Implementation:Ggml org Llama cpp Chat Peg Parser

Knowledge Sources	Ggml_org_Llama_cpp
Domains	Chat, Parsing
Last Updated	2026-02-15 00:00 GMT

Overview

Implements the PEG-based AST-to-chat-message mapping logic that converts parsed PEG syntax trees into structured chat messages with tool calls.

Description

Three mapper classes traverse PEG AST nodes by tag: `common_chat_peg_mapper` handles base reasoning/content extraction. `common_chat_peg_native_mapper` extends it to extract tool calls from nodes tagged with TOOL_OPEN, TOOL_NAME, TOOL_ID, and TOOL_ARGS, directly populating `common_chat_tool_call` objects. `common_chat_peg_constructed_mapper` handles a more granular format where individual argument names and values (string or JSON) are tagged separately, reconstructing the arguments JSON string incrementally with proper serialization and comma handling. Trailing whitespace is trimmed from extracted text.

Usage

Use this module as the output parsing backend for PEG-format chat templates. It complements the regex-based parser with a more structured approach for models whose output follows PEG-defined grammars.

Code Reference

Source Location

• Repository: Ggml_org_Llama_cpp
• File: common/chat-peg-parser.cpp
• Lines: 1-124

Signature

// Base mapper - handles reasoning and content
void common_chat_peg_mapper::from_ast(
    const common_peg_ast_arena & arena,
    const common_peg_parse_result & result);
void common_chat_peg_mapper::map(const common_peg_ast_node & node);

// Native mapper - handles tool calls with JSON arguments
void common_chat_peg_native_mapper::map(const common_peg_ast_node & node);

// Constructed mapper - handles per-argument tool calls
void common_chat_peg_constructed_mapper::map(const common_peg_ast_node & node);

// Utility
static std::string_view trim_trailing_space(std::string_view sv, int max = -1);

Import

#include "chat-peg-parser.h"
#include <nlohmann/json.hpp>

I/O Contract

Inputs

Name	Type	Required	Description
arena	common_peg_ast_arena	Yes	PEG AST arena containing the parsed syntax tree
result	common_peg_parse_result	Yes	Parse result from the PEG parser engine
node	common_peg_ast_node	Yes	Individual AST node with tag and text content

Outputs

Name	Type	Description
result.content	string	Extracted text content from CONTENT-tagged nodes
result.reasoning_content	string	Extracted reasoning text from REASONING-tagged nodes
result.tool_calls	vector<common_chat_tool_call>	Populated tool calls with name, id, and arguments fields

Usage Examples

#include "chat-peg-parser.h"

// Parse model output using PEG grammar
common_peg_arena arena = build_chat_peg_native_parser([](auto & b) {
    // define PEG grammar using builder
    return b.seq({/* grammar rules */});
});

auto parse_result = arena.parse(model_output);

// Map AST to chat message
common_chat_msg msg;
common_chat_peg_native_mapper mapper(msg);
mapper.from_ast(arena.ast_arena(), parse_result);

// msg now contains content and tool_calls

Related Pages

• Principle:Ggml_org_Llama_cpp_Chat_Parsing