Implementation:Sail sg LongSpec Math Answer Cleaners

Knowledge Sources	Sail_sg_LongSpec MetaMath
Domains	NLP, Evaluation, Mathematics
Last Updated	2026-02-14 05:00 GMT

Overview

Concrete tool for extracting and cleaning math answers from model outputs, providing answer extractors, gold answer parsers, and step-reward collators for GSM8K, MATH, and MetaMath benchmarks.

Description

The math.py module provides answer extraction functions (numeric and boxed), gold answer extractors for GSM8K and MATH datasets, answer cleaners with configurable separators, and the RAPResponseStepRewardCollator for step-level reward training. It combines MetaMath-style extraction with DeepSeek-Math answer extraction, supporting multiple prompt decomposition strategies (RAP, CoT, DeepSeek-Math CoT v2).

Usage

Import these functions when setting up evaluation post-processing for math benchmarks. The answer cleaners are passed to evaluation callback classes, and the collator is used for step-reward training data preparation.

Code Reference

Source Location

• Repository: Sail_sg_LongSpec
• File: longspec/train/data/math.py
• Lines: 1-361

Signature

def extract_answer_number(completion: str, separator: str = "The answer is: ") -> Optional[float]:
    """Extract numerical answer after a separator string."""

def gsk8k_answer_cleaner(separator: str = "The answer is: ") -> Callable:
    """Return a cleaner function for GSM8K-style numeric answers."""

def math_answer_cleaner(separator: str = "The answer is: ") -> Callable:
    """Return a cleaner function for MATH-style text answers."""

def math_gold_answer_extractor(response_field: str = "output", kv_mapping: Dict = None) -> Callable:
    """Return a function that extracts boxed gold answers from MATH dataset."""

def math_gold_answer_extractor_deepseek(query_field: str = "instruction", response_field: str = "output", kv_mapping: Dict = None) -> Callable:
    """Return a function that extracts gold answers using DeepSeek-Math extraction."""

class RAPResponseStepRewardCollator:
    def __init__(self, tokenizer: PreTrainedTokenizer, max_seq_length: int, cot_type: str = "rap"):
        """Collator for step-level reward model training with RAP/CoT decomposition."""

Import

from data.math import (
    extract_answer_number, gsk8k_answer_cleaner, math_answer_cleaner,
    math_gold_answer_extractor, RAPResponseStepRewardCollator
)

I/O Contract

Inputs

Name	Type	Required	Description
completion	str	Yes	Model-generated completion text
separator	str	No	Answer separator string (default "The answer is: ")
response_field	str	No	Key for response in data dicts
tokenizer	PreTrainedTokenizer	For collator	Tokenizer for step decomposition
max_seq_length	int	For collator	Maximum sequence length for tokenization

Outputs

Name	Type	Description
answer	str or float or None	Extracted answer (None if extraction fails)
collated_batch	Dict	Tokenized batch with labels and step endings (from collator)

Usage Examples

from data.math import gsk8k_answer_cleaner, math_gold_answer_extractor

# Clean GSM8K answer
cleaner = gsk8k_answer_cleaner(separator="The answer is: ")
result = cleaner("After calculation, The answer is: 42.")
# Returns: "42"

# Extract gold answers from MATH dataset
extractor = math_gold_answer_extractor(response_field="output")
data = [{"output": "The answer is $\\boxed{\\frac{1}{2}}$", "instruction": "Simplify 2/4"}]
data = extractor(data)
# data[0]["label"] = "\\frac{1}{2}"

Related Pages

• Environment:Sail_sg_LongSpec_Training_Environment