Implementation:FMInference FlexLLMGen Download Opt Weights
Metadata
| Field | Value |
|---|---|
| Repo | FlexLLMGen |
Domains
- Model_Preparation
- Data_Pipeline
Overview
Concrete tool for downloading and converting OPT model weights to NumPy format provided by the FlexLLMGen library.
Description
download_opt_weights() uses huggingface_hub.snapshot_download to fetch model .bin files, then iterates over all checkpoints, loading each state dict with torch.load, renaming keys (stripping "model." prefix, renaming "decoder.final_layer_norm" to "decoder.layer_norm"), and saving each parameter as a .npy file. Shared embeddings are handled by copying decoder.embed_tokens.weight to lm_head.weight.
Usage
Called automatically by OptLM.__init__ if weights are not found at the specified path. Can also be called manually via CLI:
python -m flexllmgen.opt_config --model facebook/opt-30b --path ~/opt_weights
Code Reference
| Field | Value |
|---|---|
| Source | flexllmgen/opt_config.py, Lines: 219-255 |
| Import | from flexllmgen.opt_config import download_opt_weights
|
Signature:
def download_opt_weights(model_name, path):
"""Download weights from HuggingFace Hub and convert to numpy format.
Args:
model_name: Model name (e.g., "opt-30b" or "facebook/opt-30b")
path: Local directory to store converted numpy weights
"""
I/O Contract
Inputs:
| Name | Type | Required | Description |
|---|---|---|---|
| model_name | str | Yes | OPT model name e.g. "opt-30b" |
| path | str | Yes | Local directory for weight storage |
Outputs:
None (side effect: creates {path}/{model_name}-np/ directory with .npy files for each parameter tensor)
Usage Examples
from flexllmgen.opt_config import download_opt_weights
# Download and convert OPT-30B weights
download_opt_weights("facebook/opt-30b", "~/opt_weights")
# Result: ~/opt_weights/opt-30b-np/ with individual .npy files
# Also available via CLI
# python -m flexllmgen.opt_config --model facebook/opt-30b --path ~/opt_weights