Implementation:Sail sg LongSpec Answer Extraction
| Knowledge Sources | |
|---|---|
| Domains | NLP, Evaluation, Mathematics |
| Last Updated | 2026-02-14 05:00 GMT |
Overview
Concrete tool for extracting and normalizing mathematical answers from model-generated text, originally from DeepSeek-Math.
Description
The answer_extraction module provides a comprehensive suite of functions for parsing mathematical answers from free-form text produced by language models. It handles LaTeX normalization (fractions, square roots, trigonometric functions), boxed answer extraction, program output extraction, and benchmark-specific answer parsers for MATH, GSM8K, AGIEval, OCW Courses, SAT, MMLU-STEM, MiniF2F, and CMATH. Copied from the DeepSeek-Math evaluation pipeline.
Usage
Import these functions when you need to extract the final mathematical answer from model completions during evaluation. Used as the answer extraction layer in the post-processing evaluation pipeline.
Code Reference
Source Location
- Repository: Sail_sg_LongSpec
- File: longspec/train/data/deepseek_math_utils/answer_extraction.py
- Lines: 1-355
Signature
def strip_string(string: str) -> str:
"""Normalize a LaTeX math string by removing units, fixing fracs/sqrt, etc."""
def extract_boxed_answers(text: str) -> list:
"""Extract all \\boxed{...} answers from text using brace matching."""
def extract_program_output(pred_str: str) -> str:
"""Extract output between the last ```output ... ``` block."""
def extract_answer(pred_str: str, exhaust: bool = False) -> Union[str, list]:
"""Extract answer using boxed, 'the answer is', program output, or last number fallback."""
def extract_math_answer(question: str, reasoning: str, task: str) -> list:
"""Extract math answer with support for comma-separated and 'and'-joined answers."""
def extract_math_few_shot_cot_answer(question: str, reasoning: str, task: str) -> list:
"""Extract math answer from few-shot CoT reasoning, trimming extra problems."""
def extract_last_single_answer(question: str, reasoning: str, task: str) -> str:
"""Extract the last single answer from reasoning text."""
def extract_gsm_few_shot_cot_answer(question: str, reasoning: str, task: str) -> str:
"""Extract GSM8K answer from few-shot CoT reasoning."""
Import
from data.deepseek_math_utils.answer_extraction import (
extract_answer, extract_math_answer, extract_last_single_answer,
strip_string, extract_boxed_answers
)
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| pred_str / reasoning | str | Yes | Model-generated text containing the answer |
| question | str | No | Original question text (for context-aware extraction) |
| task | str | No | Task identifier (e.g., "cot") |
| exhaust | bool | No | If True, return all found answers instead of just the last |
Outputs
| Name | Type | Description |
|---|---|---|
| answer | str or list | Extracted and normalized mathematical answer(s) |
Usage Examples
from data.deepseek_math_utils.answer_extraction import extract_answer, extract_math_answer
# Extract from boxed answer
result = extract_answer("The solution is \\boxed{42}.")
# Returns: "42"
# Extract from "the answer is" pattern
result = extract_answer("After calculation, the final answer is $3/4$. I hope it is correct.")
# Returns: "\\frac{3}{4}"
# Extract math answer with comma support
answers = extract_math_answer(
question="List all primes separated by commas",
reasoning="The answer is \\boxed{2, 3, 5}",
task="cot"
)
# Returns: ["2", "3", "5"]