Principle:Open compass VLMEvalKit Custom Prompt Building
| Source | Domain | Last Updated |
|---|---|---|
| VLMEvalKit|https://github.com/open-compass/VLMEvalKit | Vision, NLP, Software_Design | 2026-02-14 00:00 GMT |
Overview
A dispatch pattern that allows VLM adapters to override default prompt construction with model-specific formatting for individual benchmarks.
Description
Different VLMs have different optimal prompt formats for different benchmarks. VLMEvalKit implements a two-method dispatch: use_custom_prompt(dataset) returns True if the model provides custom formatting for that dataset, and build_prompt(line, dataset) constructs the model-specific prompt. During inference, the framework checks use_custom_prompt() first — if True, it calls the model's build_prompt(); otherwise, it uses the dataset's default build_prompt(). This allows models like MiniCPM-V (with 1271 lines of adapter code) to provide highly optimized prompts while simpler models use the framework defaults.
Usage
Override these methods in your VLM adapter when you need model-specific prompt formatting for certain benchmarks. Return False from use_custom_prompt() (the default) to use dataset-provided prompts.
Theoretical Basis
Strategy pattern — the prompt construction strategy is selected at runtime based on model capabilities. Dispatch is two-stage: first check if custom is needed, then delegate to the appropriate builder.