Principle:Open compass VLMEvalKit Parallel Task Execution
| Field | Value |
|---|---|
| source | Repo |
| domain | Evaluation, Distributed_Computing |
Overview
A thread-pool execution pattern that parallelizes independent I/O-bound tasks with progress tracking and incremental result saving.
Description
Many operations in VLMEvalKit are I/O-bound (API calls, file operations) and can be parallelized. The track_progress_rich() utility provides a generic parallel execution framework using Python's concurrent.futures ThreadPoolExecutor (or ProcessPoolExecutor). It manages task submission, progress tracking via tqdm, and incremental result saving to a pickle file. Each task is independently executed, and results are mapped back to their original indices. This utility is used for parallel API inference, parallel judge LLM calls, and other bulk operations.
Usage
Use when you have many independent I/O-bound tasks to execute in parallel. Common in API model inference and LLM judge evaluation.
Theoretical Basis
Thread pool pattern for I/O-bound parallelism. Tasks are embarrassingly parallel (no inter-task dependencies). Incremental saving provides fault tolerance — partial results survive crashes.