Principle:Iamhankai Forest of Thought Input Diversity Shuffling
| Knowledge Sources | |
|---|---|
| Domains | Ensemble_Methods, Data_Augmentation |
| Last Updated | 2026-02-14 03:00 GMT |
Overview
A technique that generates diverse reasoning inputs by shuffling the order of problem elements, exploiting the sensitivity of language models to input ordering.
Description
Input Diversity Shuffling creates multiple variations of the same problem by permuting the order of input elements. Since LLMs are sensitive to token ordering (e.g., "1 2 3 4" may lead to different proposals than "3 1 4 2"), shuffling naturally creates diverse reasoning paths without requiring different prompts or random seeds. Each tree in the forest receives a different input ordering, ensuring independent exploration.
Usage
Used in Game24 Forest Solving to generate distinct starting points for each tree. Also used implicitly in benchmark MCTS where different trees may see shuffled few-shot examples.
Theoretical Basis
LLMs exhibit ordering bias: the sequence of tokens in a prompt influences the generation distribution. Input shuffling exploits this:
- For n input elements, there are n! possible orderings
- Each ordering induces a different probability distribution over proposals
- Ensemble over orderings reduces the impact of any single ordering bias