Implementation:Open compass VLMEvalKit MMAlignBench
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Benchmarking, Alignment Assessment |
Overview
Benchmark dataset implementation for MMAlignBench evaluation in VLMEvalKit.
Description
MMAlignBench inherits from ImageBaseDataset and implements the MMAlignBench benchmark for multimodal alignment assessment. The TYPE field is set to 'VQA'. It uses a pairwise comparison scoring system (A>>B, A>B, A=B, B>A, B>>A) mapped to numeric scores.
Usage
Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.
Code Reference
- Source:
vlmeval/dataset/mmalignbench.py, Lines: L1-298 - Import:
from vlmeval.dataset.mmalignbench import MMAlignBench
Signature:
class MMAlignBench(ImageBaseDataset):
TYPE = 'VQA'
DATASET_URL = {...}
DATASET_MD5 = {...}
...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | TSV dataset file with image/video paths and questions |
| Outputs | Evaluation results DataFrame with scores per category |
Usage Examples
from vlmeval.dataset import build_dataset
dataset = build_dataset('MMAlignBench')
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment