Implementation:OpenGVLab InternVL HF To Custom Weight Converter

Knowledge Sources	OpenGVLab_InternVL
Domains	Weight Conversion, Model Export, InternVL
Last Updated	2026-02-07 14:00 GMT

Overview

Converts InternVL model weights from HuggingFace format back to the custom InternVL training format, with L2 distance verification against the original model.

Description

The convert_keys_back function performs the inverse mapping of the custom-to-HuggingFace conversion:

• Multi-modal projector -- multi_modal_projector.layer_norm.* maps to mlp1.0.*, linear_1.* to mlp1.1.*, linear_2.* to mlp1.3.*.
• Vision embeddings -- vision_tower.embeddings.cls_token maps to vision_model.embeddings.class_embedding, patch_embeddings.projection to patch_embedding, position_embeddings to position_embedding.
• Encoder layers -- vision_tower.encoder.layer.X maps to vision_model.encoder.layers.X, with sub-key renames: attention.projection_layer to attn.proj, layernorm_before to norm1, layernorm_after to norm2, lambda_1 to ls1, lambda_2 to ls2.
• QKV concatenation -- Separate Q, K, V projection weights and biases are buffered per layer and concatenated back into fused attn.qkv tensors.

The main script:

1. Loads the custom model configuration from the custom path.
2. Reads all .safetensors files from the HuggingFace checkpoint directory.
3. Applies convert_keys_back to transform the key names.
4. Loads the converted state dict into the model (reporting missing/unexpected keys).
5. Computes L2 distance against the original custom model using compute_l2_distance for verification.
6. Saves the converted model and tokenizer to the output path.

Usage

Use this script when you need to import HuggingFace-hosted InternVL models back into the custom training pipeline for continued fine-tuning. It is the inverse of the internvl_custom2hf.py converter.

Code Reference

Source Location

• Repository: OpenGVLab_InternVL
• File: internvl_chat/tools/internvl_hf2custom.py
• Lines: 1-182

Signature

def compute_l2_distance(model1, model2) -> float: ...

def convert_keys_back(hf_state_dict: dict) -> dict: ...

Import

from internvl_chat.tools.internvl_hf2custom import convert_keys_back

I/O Contract

Inputs

Name	Type	Required	Description
--custom_path	str	Yes	Path to custom model config and tokenizer
--hf_path	str	Yes	Path to HuggingFace-formatted safetensor weights
--save_path	str	Yes	Path to save the converted model

Outputs

Name	Type	Description
converted_model	saved model files	Model saved in custom InternVL format at save_path
tokenizer	saved tokenizer files	Tokenizer copied from custom_path to save_path
l2_distance	float	L2 distance between converted and original model (printed to stdout)

Usage Examples

Command Line Usage

python internvl_chat/tools/internvl_hf2custom.py \
    --custom_path /path/to/custom/model \
    --hf_path /path/to/hf/model \
    --save_path /path/to/output

Programmatic Usage

from internvl_chat.tools.internvl_hf2custom import convert_keys_back

# Convert HuggingFace state dict keys to custom format
custom_state_dict = convert_keys_back(hf_state_dict)

Related Pages

• Principle:OpenGVLab_InternVL_Weight_Format_Conversion