Implementation:Open compass VLMEvalKit VDC Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Video Description, Caption Quality |
Overview
Provides GPT-based evaluation utilities for the VDC (Video Detailed Captioning) benchmark with multi-dimensional scoring and description-based QA.
Description
This module defines evaluation prompts for two stages: `SYSTEM_GENER_PRED_PROMPT` / `USER_GENER_PRED_PROMPT` for generating answers from detailed descriptions, and `SYSTEM_CAL_SCORE_PROMPT` / `USER_CAL_SCORE_PROMPT` for scoring prediction-answer pairs (yes/no + 0-5 score). Dimensions are organized in `VDC_DIMENSIONS` covering short, detailed, background, main_object, camera, and overall categories for comprehensive video description assessment.
Usage
Called internally by the corresponding dataset class during evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/vdc.py, Lines: L1-156 - Import:
from vlmeval.dataset.utils.vdc import VDC_DIMENSIONS
Key Functions:
SYSTEM_CAL_SCORE_PROMPT = """..."""
USER_CAL_SCORE_PROMPT = """..."""
SYSTEM_GENER_PRED_PROMPT = """..."""
VDC_DIMENSIONS = {...}
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Video descriptions, questions, correct answers, and predicted answers |
| Outputs | Dict string with 'pred' (yes/no) and 'score' (0-5) keys |
Usage Examples
from vlmeval.dataset.utils.vdc import VDC_DIMENSIONS
print(VDC_DIMENSIONS['overall'])
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment