Implementation:Open compass VLMEvalKit VLMBias
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Benchmarking, Bias Detection |
Overview
Benchmark dataset implementation for VLMBias evaluation in VLMEvalKit.
Description
VLMBias inherits from ImageVQADataset and implements the VLMBias benchmark for evaluating bias in visual language models. It inherits the TYPE 'VQA' from its parent class. It uses OmniVerifier with GPT-4o judge for bias detection evaluation.
Usage
Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.
Code Reference
- Source:
vlmeval/dataset/vlmbias.py, Lines: L1-55 - Import:
from vlmeval.dataset.vlmbias import VLMBias
Signature:
class VLMBias(ImageVQADataset):
TYPE = 'VQA'
DATASET_URL = {...}
DATASET_MD5 = {...}
...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | TSV dataset file with image/video paths and questions |
| Outputs | Evaluation results DataFrame with scores per category |
Usage Examples
from vlmeval.dataset import build_dataset
dataset = build_dataset('VLMBias')
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment