Implementation:Mit han lab Llm awq InternVisionConfig

Knowledge Sources	Mit_han_lab_Llm_awq InternVL
Domains	Vision, Model_Configuration
Last Updated	2026-02-15 00:00 GMT

Overview

Concrete tool for configuring InternVL vision and chat model architectures provided by the tinychat framework.

Description

InternVisionConfig extends HuggingFace's PretrainedConfig to store vision transformer parameters (patch size, hidden size, number of layers, flash attention support). InternVLChatConfig is a composition config combining InternVisionConfig with either a LlamaConfig or Qwen2Config for the language model backbone, plus settings for dynamic image handling (downsample ratio, pixel shuffle version, dynamic patching).

Usage

Import these classes when instantiating InternVL3 models within the tinychat framework. InternVisionConfig is needed to configure the vision encoder, while InternVLChatConfig combines vision and language configs for the full multimodal model.

Code Reference

Source Location

• Repository: Mit_han_lab_Llm_awq
• File: tinychat/models/internvl/configuration_internvl.py
• Lines: 1-204

Signature

class InternVisionConfig(PretrainedConfig):
    model_type = 'intern_vit_6b'
    def __init__(
        self,
        num_channels=3,
        patch_size=14,
        image_size=224,
        qkv_bias=False,
        hidden_size=3200,
        num_attention_heads=25,
        intermediate_size=12800,
        qk_normalization=True,
        num_hidden_layers=48,
        use_flash_attn=True,
        hidden_act='gelu',
        norm_type='rms_norm',
        layer_norm_eps=1e-6,
        dropout=0.0,
        drop_path_rate=0.0,
        attention_dropout=0.0,
        initializer_range=0.02,
        initializer_factor=0.1,
        **kwargs,
    ):
        """Configuration for InternVisionModel (6B vision encoder)."""

class InternVLChatConfig(PretrainedConfig):
    model_type = 'internvl_chat'
    is_composition = True
    def __init__(
        self,
        vision_config=None,
        llm_config=None,
        use_backbone_lora=0,
        use_llm_lora=0,
        select_layer=-1,
        force_image_size=None,
        downsample_ratio=0.5,
        template=None,
        dynamic_image_size=False,
        use_thumbnail=False,
        ps_version='v1',
        min_dynamic_patch=1,
        max_dynamic_patch=6,
        **kwargs,
    ):
        """Composite config combining vision and language model configs."""

Import

from tinychat.models.internvl.configuration_internvl import InternVisionConfig, InternVLChatConfig

I/O Contract

Inputs

Name	Type	Required	Description
num_channels	int	No	Number of input image channels (default: 3)
patch_size	int	No	Patch size for vision embedding (default: 14)
image_size	int	No	Input image resolution (default: 224)
hidden_size	int	No	Hidden dimension of the vision transformer (default: 3200)
num_attention_heads	int	No	Number of attention heads (default: 25)
num_hidden_layers	int	No	Number of transformer encoder layers (default: 48)
use_flash_attn	bool	No	Enable Flash Attention 2 (default: True)
vision_config	dict or InternVisionConfig	No	Vision encoder configuration (for InternVLChatConfig)
llm_config	dict	No	Language model configuration (for InternVLChatConfig)
downsample_ratio	float	No	Pixel shuffle downsample ratio (default: 0.5)
template	str	No	Conversation template name

Outputs

Name	Type	Description
config	InternVisionConfig or InternVLChatConfig	Serializable configuration object compatible with HuggingFace's from_pretrained/to_dict

Usage Examples

Basic Vision Config

from tinychat.models.internvl.configuration_internvl import InternVisionConfig

# Create vision config for InternViT-6B
vision_config = InternVisionConfig(
    hidden_size=3200,
    num_attention_heads=25,
    num_hidden_layers=48,
    image_size=448,
    use_flash_attn=True,
)

Composite Chat Config

from tinychat.models.internvl.configuration_internvl import InternVisionConfig, InternVLChatConfig

# Create composite config for InternVL3
config = InternVLChatConfig(
    vision_config=InternVisionConfig().to_dict(),
    llm_config={"model_type": "qwen2", "hidden_size": 3584},
    downsample_ratio=0.5,
    template="internvl2_5",
    dynamic_image_size=True,
    max_dynamic_patch=12,
)

Related Pages

• Principle:Mit_han_lab_Llm_awq_Multimodal_Model_Configuration