Principle:Iamhankai Forest of Thought Tree of Thought Search
| Knowledge Sources | |
|---|---|
| Domains | Search_Algorithms, Reasoning |
| Last Updated | 2026-02-14 03:00 GMT |
Overview
A deliberate reasoning framework that decomposes problems into intermediate steps, generates multiple candidates at each step, evaluates them via LLM scoring, and searches for optimal reasoning paths using DFS or BFS.
Description
Tree of Thoughts (ToT) structures LLM reasoning as a tree search over intermediate "thoughts" (reasoning steps). Unlike Chain-of-Thought which follows a single linear path, ToT generates multiple candidate next steps at each decision point, evaluates them using an LLM value function, and uses search algorithms (DFS or BFS) to navigate the thought space. This enables backtracking from dead-end reasoning paths and systematic exploration of alternatives.
In the FoT framework, ToT serves as one of three base reasoning modes. Each tree in the forest can use ToT to explore step-by-step solutions to math problems.
Usage
Use this principle when the base search mode is set to tot. ToT is suited for problems that benefit from step-by-step decomposition where each step can be independently evaluated, such as multi-step math proofs or structured problem-solving.
Theoretical Basis
ToT operates on a tree where:
- Nodes represent partial solutions (accumulated reasoning steps)
- Edges represent individual thought steps
- Values are LLM-assigned scores estimating solution quality
Search algorithms:
- DFS: Explores depth-first with backtracking when values fall below end_gate threshold
- BFS: Generates all candidates at each level, prunes to top-k by value, proceeds to next level
Pseudo-code:
# Abstract ToT algorithm
def tot_search(problem, max_depth, branch_factor):
root = Node(problem)
for depth in range(max_depth):
candidates = generate_next_steps(root, branch_factor)
values = evaluate_steps(candidates)
root = select_best(candidates, values)
return extract_summary(root)