Implementation:Open compass VLMEvalKit BaseAPI

Field	Value
source	VLMEvalKit
domain	Vision, Model_Architecture, API_Integration

Overview

Concrete abstract base class for all API-based VLM wrappers in VLMEvalKit providing retry logic and a common inference interface.

Description

BaseAPI in vlmeval/api/base.py provides the retry-enabled interface for API models. Constructor accepts retry (default 10), wait (default 1s), system_prompt, verbose, and fail_msg. generate() preprocesses input, then calls generate_inner() up to retry times with randomized wait between attempts. chat() supports multi-turn with automatic conversation truncation. working() tests if the API is responsive. The is_api attribute is implicitly True for BaseAPI subclasses.

Usage

Subclass when adding a new API provider. Implement generate_inner(inputs, **kwargs) returning (ret_code, answer, log).

Code Reference

Source: vlmeval/api/base.py, Lines: L9-300
Import: from vlmeval.api.base import BaseAPI

Signature:

class BaseAPI:
    allowed_types = ['text', 'image', 'video']
    INTERLEAVE = True
    INSTALL_REQ = False

    def __init__(
        self,
        retry: int = 10,
        wait: int = 1,
        system_prompt: Optional[str] = None,
        verbose: bool = True,
        fail_msg: str = 'Failed to obtain answer via API.',
        **kwargs
    ): ...

    @abstractmethod
    def generate_inner(self, inputs, **kwargs) -> Tuple[int, str, str]: ...

    def generate(self, message, **kwargs) -> str: ...

    def chat(self, messages, **kwargs) -> str: ...

    def working(self) -> bool: ...

I/O Contract

Direction	Description
Inputs	message — `str`, `dict`, `List[str]`, or `List[Dict]`; **kwargs passed to `generate_inner`
Outputs	`generate()` returns `str` (prediction or `fail_msg`); `generate_inner()` must return `(int, str, str)` tuple

Usage Examples

from vlmeval.api.base import BaseAPI

class MyAPIModel(BaseAPI):
    def __init__(self, model_name, **kwargs):
        super().__init__(retry=10, wait=2, **kwargs)
        self.model_name = model_name

    def generate_inner(self, inputs, **kwargs):
        # inputs is List[Dict] with type/value
        try:
            response = call_my_api(self.model_name, inputs)
            return 0, response, "success"
        except Exception as e:
            return -1, str(e), str(e)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment