Implementation:Open compass VLMEvalKit Track Progress Rich

Field	Value
source	VLMEvalKit
domain	Evaluation, Distributed_Computing

Overview

Concrete tool for parallel task execution with progress tracking and incremental result saving provided by VLMEvalKit.

Description

track_progress_rich() in vlmeval/utils/mp_util.py (L31-75) provides parallel execution using ThreadPoolExecutor (default) or ProcessPoolExecutor. Takes a callable function, list of task inputs (each a dict of kwargs), and optional save path. Wraps each task with index tracking, submits all to the executor, tracks completion via tqdm progress bar, and saves results incrementally to a pickle file if save path and keys are provided. Returns list of results in original order.

Usage

Used internally by infer_data_api() for parallel API inference. Can be used directly for any bulk parallel operation.

Code Reference

• Source: vlmeval/utils/mp_util.py, Lines: L31-75
• Signature:

def track_progress_rich(
    func: Callable,              # Function to call for each task
    tasks: Iterable = tuple(),   # List of task inputs (dicts)
    nproc: int | None = 1,       # Number of parallel workers
    save=None,                   # Path to save incremental results
    keys=None,                   # Keys for result mapping
    use_process: bool = False,   # Use ProcessPoolExecutor instead
    **kwargs
) -> list:
    """
    Execute func on each task in parallel with progress tracking.
    Returns list of results in original order.
    """

• Import: from vlmeval.utils import track_progress_rich

I/O Contract

Direction	Name	Type	Description
Input	func	Callable	Function to call for each task
Input	tasks	Iterable of dicts	List of task inputs as keyword argument dicts
Input	nproc	int	Number of parallel workers
Input	save	str path	Path to save incremental results
Input	keys	list	Keys for result mapping
Output	results	list	List of results in original task order
Side Effect	—	pickle file	Saves incremental results to pickle file at save path

Usage Examples

from vlmeval.utils import track_progress_rich

# Parallel API calls
def call_api(message, dataset):
    return model.generate(message=message, dataset=dataset)

tasks = [dict(message=msg, dataset="MMBench") for msg in prompts]
results = track_progress_rich(
    call_api,
    tasks,
    nproc=8,
    save="partial_results.pkl",
    keys=list(range(len(tasks)))
)

Related Pages

• Principle:Open_compass_VLMEvalKit_Parallel_Task_Execution
• Environment:Open_compass_VLMEvalKit_Python_Runtime_Environment