Principle:FMInference FlexLLMGen HELM Dependency Installation
| Field | Value |
|---|---|
| Sources | HELM Benchmark, FlexLLMGen |
| Domains | Environment_Setup, Benchmark_Integration |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
An environment preparation step that installs the Stanford HELM benchmark framework at a specific pinned version for compatibility with FlexLLMGen's evaluation integration.
Description
FlexLLMGen's HELM integration requires crfm-helm version 0.2.1 specifically. Newer versions may have breaking API changes in the benchmark runner, adapter, and metric interfaces. The installation step ensures the correct version is available before running evaluation workflows.
Usage
Install crfm-helm==0.2.1 before running any HELM benchmark evaluation with FlexLLMGen.
Theoretical Basis
Pinning dependency versions ensures reproducible evaluation results. HELM's API has evolved significantly across versions, and FlexLLMGen's wrapper code targets the 0.2.1 interface specifically.
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment