Implementation:Princeton nlp Tree of thought llm Gpt Completion

Knowledge Sources	tree-of-thought-llm OpenAI API Reference
Domains	API_Design, NLP, Infrastructure
Last Updated	2026-02-14 03:30 GMT

Overview

Concrete tool for making OpenAI ChatCompletion API calls with backoff retry and token tracking provided by the Tree of Thoughts framework.

Description

The gpt function is the unified entry point for all LLM calls in the framework. It wraps a user prompt into the ChatCompletion message format and delegates to chatgpt, which handles batching (max 20 per request), retry via backoff.on_exception with exponential backoff on openai.error.OpenAIError, and global token usage accumulation.

Usage

Import gpt whenever you need to call the LLM from within any module (task evaluation, thought generation, voting). In practice, gpt is monkey-patched with functools.partial in solve() and naive_solve() to bind the model and temperature for the duration of an experiment.

Code Reference

Source Location

Repository: tree-of-thought-llm
File: src/tot/models.py
Lines: 1-47

Signature

def gpt(prompt, model="gpt-4", temperature=0.7, max_tokens=1000, n=1, stop=None) -> list:
    """
    Send a prompt to the OpenAI ChatCompletion API.

    Args:
        prompt (str): The user prompt text.
        model (str): Model identifier (default 'gpt-4').
        temperature (float): Sampling temperature (default 0.7).
        max_tokens (int): Max tokens per completion (default 1000).
        n (int): Number of completions to generate (default 1).
        stop (str or list or None): Stop sequence(s) (default None).

    Returns:
        list[str]: List of n completion strings.
    """

Import

from tot.models import gpt

I/O Contract

Inputs

Name	Type	Required	Description
prompt	str	Yes	User prompt text to send to the LLM
model	str	No	Model identifier (default 'gpt-4')
temperature	float	No	Sampling temperature (default 0.7)
max_tokens	int	No	Maximum tokens per completion (default 1000)
n	int	No	Number of completions to generate (default 1). Batched internally in groups of 20.
stop	str/list/None	No	Stop sequence(s) (default None)

Outputs

Name	Type	Description
return	list[str]	List of completion strings, one per requested sample

Usage Examples

Basic Single Completion

from tot.models import gpt

# Single completion with default settings
responses = gpt("What is 2 + 2?")
print(responses[0])  # "4"

Multiple Samples with Custom Temperature

from tot.models import gpt

# Generate 5 completions at high temperature
responses = gpt(
    "Use numbers 1 2 3 4 to make 24",
    model="gpt-4",
    temperature=1.0,
    n=5,
    stop="\n"
)
for r in responses:
    print(r)

Monkey-Patching in BFS Solve

from functools import partial
from tot.models import gpt

# Bind model and temperature for the experiment duration
gpt_configured = partial(gpt, model="gpt-4", temperature=0.7)
result = gpt_configured("Your prompt here", n=3)

Related Pages

Implements Principle

Principle:Princeton_nlp_Tree_of_thought_llm_LLM_API_Wrapping

Requires Environment

Environment:Princeton_nlp_Tree_of_thought_llm_Python_OpenAI

Uses Heuristic

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment