Principle:Iamhankai Forest of Thought Answer Extraction
| Knowledge Sources | |
|---|---|
| Domains | NLP, Evaluation |
| Last Updated | 2026-02-14 03:00 GMT |
Overview
A pattern for extracting structured answer labels from unstructured LLM reasoning output, handling diverse answer formats across math benchmarks.
Description
Answer Extraction transforms free-form LLM output (which may contain reasoning steps, explanations, and formatting artifacts) into a clean answer label suitable for comparison. Different benchmarks use different answer formats: GSM8K uses #### delimiters, MATH uses \\boxed{} LaTeX formatting, and AIME uses plain numeric answers. The extraction pipeline must handle all these formats and clean LaTeX artifacts (dollar signs, \\text{}, etc.).
This is a critical preprocessing step because LLM outputs are verbose and inconsistently formatted. Without proper extraction, even correct answers would fail validation.
Usage
Used throughout FoT whenever LLM output needs to be converted to a comparable answer label. Applied in tree-level answer tracking, forest consensus voting, and CGDM post-processing.
Theoretical Basis
The extraction follows a priority-based pattern matching approach:
# Abstract extraction logic
def extract(text, dataset):
if dataset == "gsm8k":
return extract_after_delimiter(text, "####")
else:
boxed = extract_boxed(text)
if boxed:
return clean_latex(boxed)
return extract_last_number(text)
Format priority:
- \\boxed{} content (highest priority, most explicit)
- Delimiter-based extraction (####, The answer is)
- Last numeric value in the text (fallback)