Principle:SqueezeAILab ETS Hyperparameter Configuration
| Knowledge Sources | |
|---|---|
| Domains | Configuration, Experiment_Design |
| Last Updated | 2026-02-14 02:00 GMT |
Overview
A configuration management pattern that externalizes search hyperparameters into YAML files to enable reproducible experiment sweeps.
Description
The ETS algorithm's behavior is controlled by a set of hyperparameters that govern the search width, node selection strategy, temperature settings, cost and diversity penalties, and token generation limits. Rather than hard-coding these values, they are externalized into YAML configuration files that are loaded at runtime. This enables:
- Experiment sweeps: Running the same algorithm with different parameter settings (e.g., width=16, 64, 256)
- Reproducibility: Each experiment's configuration is fully captured in a single file
- Separation of concerns: Algorithm code remains clean of configuration details
The key hyperparameters include:
- width: Total search budget (number of candidate trajectories)
- select_method: Node selection strategy (softmax_costmodel, beam_search, dvts, softmax)
- lambdac: Cost penalty weight in the ILP objective
- lambdas: Diversity penalty weight (0 disables diversity enforcement)
- softmax_temperature: Temperature for softmax-based width allocation
- temperature: Sampling temperature for text generation
Usage
Create a YAML config file before running any ETS experiment. The config file path is passed to rebase.py via the --parameter_path CLI argument. Different configs enable sweeping over search budgets and hyperparameter settings.
Theoretical Basis
Configuration-driven experimentation follows the principle of parametric control: by varying hyperparameters systematically while holding the algorithm fixed, researchers can study the sensitivity and scaling behavior of the search method. In the ETS context:
- Width scaling: Increasing width from 16 to 256 tests how accuracy improves with more compute
- Lambda tuning: The cost penalty (lambdac) and diversity penalty (lambdas) control the trade-off between exploration and KV cache efficiency
- Temperature control: Softmax temperature governs how aggressively the search concentrates budget on high-scoring nodes
Pseudo-code:
# Abstract config loading pattern
config = load_yaml("experiment_config.yaml")
tree = Tree(root, config, reward_model, embedding_model)
# config["select_method"] determines which selection algorithm runs
# config["width"] sets total compute budget