Implementation:Open compass VLMEvalKit Infer Data Job Video

Field	Value
source	VLMEvalKit
domain	Vision, Video_Understanding, Distributed_Computing

Overview

Concrete tool for orchestrating distributed video inference with frame sampling and pack mode support provided by VLMEvalKit.

Description

infer_data_job_video() in vlmeval/inference_video.py orchestrates video benchmark inference. It delegates to infer_data() (video variant) which extracts frames from videos via dataset.save_video_frames(), builds video prompts, and runs model generation. On rank 0, it merges results and handles MMBench-Video pack mode specially via dataset.load_pack_answers().

Usage

Called by run.py when dataset.MODALITY == 'VIDEO'.

Code Reference

• Source: vlmeval/inference_video.py, Lines: L207-259
• Signature:

def infer_data_job_video(
    model,                    # VLM instance or name
    work_dir: str,            # Output directory
    model_name: str,          # Model name for file naming
    dataset,                  # VideoBaseDataset subclass
    result_file_name: str,    # Result filename
    verbose: bool = False,
    api_nproc: int = 4,
    use_vllm: bool = False
) -> model:
    """
    Orchestrates distributed video inference.
    Returns model instance for reuse.
    """

• Import: from vlmeval.inference_video import infer_data_job_video

I/O Contract

Inputs

Parameter	Type	Description
model	Union[str, BaseModel]	VLM instance or model name
work_dir	str	Output directory
model_name	str	Model name for file naming
dataset	VideoBaseDataset	Video dataset subclass
result_file_name	str	Result filename
verbose	bool	Print predictions (default False)
api_nproc	int	Parallel API threads (default 4)
use_vllm	bool	Enable vLLM acceleration (default False)

Outputs

Output	Type	Description
Returns	model instance	The model instance for reuse
Side effect	Prediction file	File with video QA results

Usage Examples

from vlmeval.inference_video import infer_data_job_video
from vlmeval.dataset import build_dataset

dataset = build_dataset("MVBench_8frame")
model = infer_data_job_video(
    model="Video-LLaVA-7B",
    work_dir="./results",
    model_name="Video-LLaVA-7B",
    dataset=dataset,
    result_file_name="Video-LLaVA-7B_MVBench_8frame.xlsx"
)

Related Pages

• Principle:Open_compass_VLMEvalKit_Video_Inference_Orchestration
• Environment:Open_compass_VLMEvalKit_GPU_CUDA_Environment
• Heuristic:Open_compass_VLMEvalKit_WORLD_SIZE_Unset_For_Model_Building
• Heuristic:Open_compass_VLMEvalKit_SKIP_ERR_For_Graceful_Inference
• Heuristic:Open_compass_VLMEvalKit_Video_Frame_Sampling_Configuration