Principle:Open compass VLMEvalKit Dataset Registration
| Field | Value |
|---|---|
| source | Repo |
| domain | Vision, Evaluation, Software_Design |
Overview
A registration pattern for adding new benchmark datasets to VLMEvalKit's dataset discovery system.
Description
After implementing a dataset class, it must be registered so that build_dataset() can discover it. Registration involves:
- Implement the dataset class with
DATASET_URLandDATASET_MD5class attributes - Import the class in
vlmeval/dataset/__init__.py - Add the class to the appropriate list (
IMAGE_DATASET,VIDEO_DATASET, orCUSTOM_DATASET)
The framework then auto-discovers supported dataset names via each class's supported_datasets() classmethod and populates the SUPPORTED_DATASETS list. For video datasets, an additional step registers partial() entries in supported_video_datasets in vlmeval/dataset/video_dataset_config.py.
Usage
Follow this pattern when adding any new benchmark to VLMEvalKit.
Theoretical Basis
Auto-discovery registration — classes declare their supported names, the framework aggregates them into a central lookup. Similar to plugin registration.
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment