Principle:Ollama Ollama Tool Call Parsing
| Knowledge Sources | |
|---|---|
| Domains | Parsing, Tool Calling |
| Last Updated | 2025-02-15 00:00 GMT |
Overview
Tool Call Parsing extracts structured tool invocation data from the raw output stream of a language model, handling incremental token delivery, model-specific formatting conventions, and partial parse states to produce well-formed tool call objects in real time.
Core Concepts
Streaming Parse Challenge
Unlike batch parsing where the complete text is available, tool call parsing must operate on a token-by-token stream. The parser receives text incrementally and must detect the beginning of a tool call, accumulate the tool call content, detect the end of the tool call, and emit the parsed result. This requires maintaining parse state across token boundaries, as a tool call's opening delimiter, function name, and arguments may span many tokens.
Format-Specific Parsers
Different model families emit tool calls in different formats. Some models produce JSON directly (e.g., {"name": "func", "arguments": {...}}), others use XML-like tags (e.g., <tool_call>...</tool_call>), and some use specialized markup (e.g., <|tool_call|>). Each format requires a dedicated parser that understands its delimiters and structure. The parsing system selects the appropriate parser based on the model's template configuration.
Incremental JSON Parsing
For models that emit tool calls as JSON, the parser must handle incremental JSON construction. As tokens arrive, the parser builds up a JSON string, tracking brace depth to determine when a complete JSON object has been received. This incremental approach avoids buffering the entire response and allows early detection of malformed tool calls.
Content vs. Tool Call Separation
A model's output may contain both regular text content and tool calls interleaved. The parser must cleanly separate these segments, routing regular text to the content stream and tool call data to the tool call stream. This separation is critical for the API layer, which must present content and tool calls as distinct fields in the response.
Error Recovery
When the model produces malformed tool call syntax, the parser must recover gracefully. This may involve treating the malformed tool call as regular text content, logging a warning, or attempting to extract partial information. Robust error recovery is essential because model output is inherently unpredictable and cannot be guaranteed to conform to the expected tool call format.
Implementation Notes
The core tool call parsing logic is in model/parsers/, which provides parser implementations for different tool call formats. The template system in tools/template.go associates each model family with its expected tool call format and parser. The runner's output processing pipeline invokes the appropriate parser on the generation stream, accumulating tool call objects and emitting them alongside regular content in the streamed response.