Implementation:Open compass VLMEvalKit Check Command

Source	Domain
VLMEvalKit	Vision, Testing

Overview

Concrete tool for validating VLM adapter integration by running test generations via the vlmutil CLI.

Description

CHECK() in vlmeval/tools.py (L324-337) instantiates a model from supported_VLM, then runs 4 test generations:

1. Plain text list with image path.
2. Dict-format messages with image.
3. Multi-image plain text list.
4. Multi-image dict format.

Tests 3 and 4 are skipped if model.INTERLEAVE is False. Each test result is printed. For model group names, recursively checks all models in the group.

Usage

Run vlmutil check MyModel-7B from the command line to validate a model adapter.

Code Reference

• Source: vlmeval/tools.py, Lines: L324-337 (CHECK function), L603-607 (CLI handler)
• Signature:

def CHECK(val: str) -> None:
    """
    Args:
        val: Model name (key in supported_VLM) or model group name.
    Prints:
        Test results for each input format.
    """

• Import: from vlmeval.tools import CHECK (or via CLI: vlmutil check model_name)

I/O Contract

Parameter	Type	Description
val	str	Model name to check (key in supported_VLM)
Output	None	Printed test results; no return value

Usage Examples

# CLI usage
vlmutil check MiniCPM-V-2_6

# Output:
# Model: MiniCPM-V-2_6
# Test 1: This image shows a red apple on a white background.
# Test 2: This image shows a red apple on a white background.
# Test 3: There are 2 apples in these images.
# Test 4: There are 2 apples in these images.

Related Pages

• Principle:Open_compass_VLMEvalKit_Model_Validation
• Environment:Open_compass_VLMEvalKit_GPU_CUDA_Environment