Implementation:Openai Evals LangChainMathChainCompletionFn

Knowledge Sources	Openai_Evals
Domains	Evaluation, Math Reasoning
Last Updated	2026-02-14 10:00 GMT

Overview

Concrete completion function for solving math problems via LangChain's LLMMathChain, provided by the evals library.

Description

This module defines two classes that integrate LangChain into the OpenAI evals framework. LangChainCompletionResult is a thin wrapper around a LangChain response string that conforms to the CompletionResult interface, stripping whitespace before returning completions. LangChainMathChainCompletionFn is the main completion function: it instantiates an OpenAI LLM with temperature 0 and wraps it inside LangChain's LLMMathChain. When called with a prompt, it converts the prompt via CompletionPrompt, runs the math chain, strips the leading "Answer: " prefix from the LangChain output, records the sampling event, and returns a LangChainCompletionResult. This enables evaluating LangChain-based math reasoning against standard evals benchmarks.

Usage

Import LangChainMathChainCompletionFn when you want to register a completion function that delegates mathematical problem solving to LangChain's LLMMathChain. This is typically referenced in an eval's YAML configuration as a custom completion function. The OPENAI_API_KEY environment variable must be set.

Code Reference

Source Location

• Repository: Openai_Evals
• File: evals/completion_fns/langchain_math.py
• Lines: 1-31

Signature

class LangChainCompletionResult(CompletionResult):
    def __init__(self, response) -> None:
        ...
    def get_completions(self) -> list[str]:
        ...

class LangChainMathChainCompletionFn(CompletionFn):
    def __init__(self, **kwargs) -> None:
        ...
    def __call__(self, prompt, **kwargs) -> LangChainCompletionResult:
        ...

Import

from evals.completion_fns.langchain_math import LangChainMathChainCompletionFn

I/O Contract

Inputs

LangChainCompletionResult.__init__

Name	Type	Required	Description
response	str	Yes	Raw string response returned by the LangChain LLMMathChain

LangChainMathChainCompletionFn.__init__

Name	Type	Required	Description
**kwargs	Any	No	Additional keyword arguments (currently unused; reserved for future extension)

LangChainMathChainCompletionFn.__call__

Name	Type	Required	Description
prompt	Union[str, list]	Yes	The prompt to send to the math chain, converted via CompletionPrompt
**kwargs	Any	No	Additional keyword arguments (passed through but not used by the math chain)

Outputs

LangChainCompletionResult.get_completions

Name	Type	Description
completions	list[str]	Single-element list containing the stripped response string

LangChainMathChainCompletionFn.__call__

Name	Type	Description
result	LangChainCompletionResult	Wrapper around the math chain answer with the "Answer:" prefix removed

Usage Examples

from evals.completion_fns.langchain_math import LangChainMathChainCompletionFn

# Instantiate the completion function (requires OPENAI_API_KEY env var)
math_fn = LangChainMathChainCompletionFn()

# Solve a math problem
result = math_fn(prompt="What is the square root of 144?")
answers = result.get_completions()
print(answers)  # e.g. ['12.0']

Related Pages

• Environment:Openai_Evals_Python_Runtime
• Implementation:Openai_Evals_CompletionFn_Protocol