Implementation:Open compass VLMEvalKit Infer Data Job

Field	Value
source	VLMEvalKit\|https://github.com/open-compass/VLMEvalKit
domain	Vision, Evaluation, Distributed_Computing

Overview

Concrete tool for orchestrating distributed image inference with checkpoint/resume support provided by VLMEvalKit.

Description

infer_data_job() manages the complete inference lifecycle: loads prior results from existing result files, creates per-rank pickle files, delegates to infer_data() for actual inference, synchronizes via dist.barrier(), and merges all rank outputs on rank 0. Supports SPLIT_THINK environment variable for separating thinking/reasoning content from predictions. Handles both local models and API models transparently.

Usage

Called by run.py for each model x dataset pair. Use when running inference on image benchmarks with local VLMs or API models.

Code Reference

Source: vlmeval/inference.py, Lines: L188-268
Signature:

def infer_data_job(
    model,               # VLM instance or model name string
    work_dir: str,       # Output directory for results
    model_name: str,     # Model name for file naming
    dataset,             # Dataset instance (ImageBaseDataset subclass)
    verbose: bool = False,
    api_nproc: int = 4,  # Parallel threads for API models
    ignore_failed: bool = False,
    use_vllm: bool = False
) -> model:
    """
    Orchestrates distributed inference across GPU ranks.
    Returns the model instance (for reuse across datasets).
    """

Import: from vlmeval.inference import infer_data_job

I/O Contract

Inputs

Parameter	Type	Description
model	Union[str, BaseModel]	Model instance or name (instantiated from supported_VLM if string)
work_dir	str	Directory for result files
model_name	str	Model name for file naming
dataset	ImageBaseDataset	Dataset with .data DataFrame and .build_prompt() method
verbose	bool	Print predictions (default False)
api_nproc	int	Parallel API threads (default 4)
ignore_failed	bool	Skip failed predictions on resume (default False)
use_vllm	bool	Enable vLLM acceleration (default False, limited to Llama4/Qwen2-VL)

Outputs

Output	Type	Description
Returns	model instance (BaseModel or BaseAPI)	The model instance for reuse across datasets
Side effect	Result file	File at {work_dir}/{model_name}_{dataset_name}.{format} with columns: index, question, prediction, answer, etc.

Usage Examples

from vlmeval.inference import infer_data_job
from vlmeval.dataset import build_dataset
from vlmeval.config import supported_VLM

# Build dataset and run inference
dataset = build_dataset("MMBench_DEV_EN_V11")
model = infer_data_job(
    model="InternVL2-8B",
    work_dir="./results",
    model_name="InternVL2-8B",
    dataset=dataset
)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment