Implementation:Openai Evals TogetherSolver

Knowledge Sources	Openai_Evals
Domains	Evaluation, LLM Provider Integration
Last Updated	2026-02-14 10:00 GMT

Overview

Concrete solver for running evaluation tasks against the Together AI API provided by the evals library.

Description

TogetherSolver is an OpenAISolver subclass that adapts the OpenAI-compatible completion interface for the Together AI platform, which hosts a variety of open-source models (e.g. Llama-2, Mixtral). Because Together's API is largely OpenAI-compatible, the solver inherits most behaviour from OpenAISolver and overrides only the properties and methods that differ.

The module also defines a standalone helper function is_chat_model that checks whether a given model identifier corresponds to a chat-capable model on the Together platform.

Key overrides and behaviours:

API base URL -- The _api_base property returns "https://api.together.xyz/v1", redirecting all requests from OpenAI's endpoint to Together's.
API key -- The _api_key property reads from the TOGETHER_API_KEY environment variable instead of OPENAI_API_KEY.
Chat model detection -- _is_chat_model delegates to the module-level is_chat_model() function, which maintains a hard-coded set of known Together chat models (e.g. meta-llama/Llama-2-70b-chat-hf, mistralai/Mixtral-8x7B-Instruct-v0.1). Unsupported models raise NotImplementedError.
Message processing -- The _process_msgs method reformats the conversation history to comply with open-source model expectations: system messages after the first position are converted to user messages; when the first message is system and the second is assistant, the system message is also converted to user for consistency. An optional merge_adjacent_msgs flag (set at init) merges consecutive same-role messages by concatenating their content with double newlines.
No valid_answers support -- The constructor raises NotImplementedError if valid_answers is set, since Together's API does not support constrained decoding via logit_bias.
No prechecks -- _perform_prechecks returns None unconditionally because Together does not expose a tokenizer for context-length estimation.
No completion options preprocessing -- _preprocess_completion_fn_options is a no-op since valid_answers is unsupported.
Error handling -- The _completion_exception property returns PermissionDeniedError (Together uses a different error code for context-length violations). _handle_completion_exception catches invalid_request_error responses and returns a SolverResult with the error message rather than crashing.

Usage

Import TogetherSolver when you need to evaluate open-source models hosted on the Together AI platform. It is typically specified by class path in YAML eval configurations. Requires the TOGETHER_API_KEY environment variable.

Code Reference

Source Location

Repository: Openai_Evals
File: evals/solvers/providers/together/together_solver.py
Lines: 1-146

Signature

def is_chat_model(model: str) -> bool:

class TogetherSolver(OpenAISolver):
    def __init__(self, merge_adjacent_msgs: bool = False, **kwargs):
    @property
    def _api_base(self) -> Optional[str]:
    @property
    def _api_key(self) -> Optional[str]:
    @property
    def _completion_exception(self) -> Exception:
    def _is_chat_model(self, model: str) -> bool:
    def _preprocess_completion_fn_options(self) -> dict:
    def _perform_prechecks(self, msgs: list[dict[str, str]]) -> Optional[SolverResult]:
    def _process_msgs(self, msgs: list[dict[str, str]]) -> list[dict[str, str]]:
    def _handle_completion_exception(self, e: Exception) -> SolverResult:

Import

from evals.solvers.providers.together.together_solver import TogetherSolver

I/O Contract

Inputs

Name	Type	Required	Description
merge_adjacent_msgs	`bool`	No (default `False`)	When `True`, consecutive messages with the same role are merged by concatenating content with double newlines. Useful for models that reject non-alternating turns.
**kwargs	`dict`	Varies	All remaining keyword arguments are forwarded to OpenAISolver.__init__, including `completion_fn_options` (with `model`, `temperature`, etc.), `postprocessors`, and `registry`.
task_state	`TaskState`	Yes (at solve time)	The evaluation task state containing `task_description` and `messages`.

Outputs

Name	Type	Description
result	`SolverResult`	Contains the model's text response in `output`. If a context-length or API error occurs, `output` holds the error message and `error` holds the error body.

Usage Examples

from evals.solvers.providers.together.together_solver import TogetherSolver
from evals.task_state import TaskState, Message

# Instantiate the solver for a Llama-2 chat model
solver = TogetherSolver(
    merge_adjacent_msgs=True,
    completion_fn_options={
        "model": "meta-llama/Llama-2-70b-chat-hf",
        "extra_options": {"temperature": 0.7, "max_tokens": 512},
    },
)

# Build a task state
task_state = TaskState(
    task_description="You are a helpful coding assistant.",
    messages=[
        Message(role="user", content="Write a Python function to reverse a string."),
    ],
)

# Solve the task
result = solver(task_state)
print(result.output)

# Check model type
from evals.solvers.providers.together.together_solver import is_chat_model
print(is_chat_model("meta-llama/Llama-2-70b-chat-hf"))  # True

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment