Implementation:FMInference FlexLLMGen HELM Install

Field	Value
Sources	FlexLLMGen, HELM Benchmark
Domains	Environment_Setup, Benchmark_Integration
Last Updated	2026-02-09 00:00 GMT

Overview

External tool documentation for installing the HELM benchmark framework for FlexLLMGen evaluation.

Description

This External Tool Doc covers installing crfm-helm at the pinned version 0.2.1 required by FlexLLMGen's HELM integration. The helm_run.py file header documents the required version.

Code Reference

• Source: flexllmgen/apps/helm_run.py, Lines: 1-6 (version comment)
• Import: No Python import for installation (shell command)

Installation command:

pip install crfm-helm==0.2.1

Verification:

import helm
# Verify helm.benchmark, helm.common, helm.proxy modules are available

I/O Contract

Direction	Name	Description
Input	Python environment	Python environment with FlexLLMGen already installed
Input	Internet access	Required for downloading from PyPI
Output	crfm-helm package	crfm-helm package installed with helm.benchmark, helm.common, helm.proxy modules available

Usage Examples

# Install HELM (pinned version)
pip install crfm-helm==0.2.1

# Verify installation
python -c "from helm.benchmark.runner import create_scenario; print('HELM OK')"

Related Pages

• Principle:FMInference_FlexLLMGen_HELM_Dependency_Installation