Implementation:Openai Evals Registry Make Completion Fn

Knowledge Sources	OpenAI Evals
Domains	Evaluation, Model_Integration
Last Updated	2026-02-14 10:00 GMT

Overview

Concrete tool for resolving a string identifier to an instantiated CompletionFn provided by the evals Registry class.

Description

The Registry.make_completion_fn method implements a multi-strategy resolution chain. It first checks for the "dummy" sentinel, then attempts to match known OpenAI model names (using is_chat_model and api_model_ids lookups), and finally falls back to YAML registry lookup across both completion_fns and solvers namespaces. The method automatically determines context window size via n_ctx_from_model_name and can accept additional keyword arguments that are merged into the spec.

Usage

Use this method when you need to instantiate a CompletionFn from a name string. This is called internally by the oaieval CLI when processing the first positional argument, but can also be called directly when programmatically running evaluations.

Code Reference

Source Location

Repository: openai/evals
File: evals/registry.py (lines 120-151)

Signature

class Registry:
    def make_completion_fn(
        self,
        name: str,
        **kwargs: Any,
    ) -> CompletionFn:
        """
        Create a CompletionFn. The name can be one of the following formats:
        1. openai-model-id (e.g. "gpt-3.5-turbo")
        2. completion-fn-id (from the registry)

        Args:
            name: Model ID or registered completion function name.
            **kwargs: Additional arguments passed to the CompletionFn constructor.

        Returns:
            Instantiated CompletionFn ready for evaluation.

        Raises:
            ValueError: If no CompletionFn or Solver found with the given name.
        """

Import

from evals.registry import Registry

I/O Contract

Inputs

Name	Type	Required	Description
name	str	Yes	Model ID (e.g. "gpt-3.5-turbo") or registered completion function name
**kwargs	Any	No	Additional arguments merged into spec args (e.g. temperature, max_tokens)

Outputs

Name	Type	Description
return value	CompletionFn	Instantiated completion function ready for evaluation

Usage Examples

Resolve an OpenAI Model

from evals.registry import Registry

registry = Registry()

# Resolve a chat model by name
completion_fn = registry.make_completion_fn("gpt-3.5-turbo")

# Resolve with additional arguments
completion_fn = registry.make_completion_fn(
    "gpt-4",
    temperature=0.0,
    max_tokens=500,
)

Resolve a Custom Registered Function

from evals.registry import Registry

registry = Registry()

# Resolve a YAML-registered completion function
completion_fn = registry.make_completion_fn("my-custom-fn")

# Resolve with custom registry paths
registry.add_registry_paths(["./my_registry"])
completion_fn = registry.make_completion_fn("my-fn")

Resolve a Dummy for Testing

from evals.registry import Registry

registry = Registry()

# Returns a DummyCompletionFn for testing without API calls
completion_fn = registry.make_completion_fn("dummy")
result = completion_fn("Hello")
print(result.get_completions())  # ["This is a dummy response."]

Related Pages

Implements Principle

Principle:Openai_Evals_Completion_Function_Resolution

Requires Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment