Principle:Open compass VLMEvalKit Environment Setup

Field	Value
Source	https://github.com/open-compass/VLMEvalKit
Domain	Vision, Evaluation
Last Updated	2026-02-14 00:00 GMT

Overview

A configuration pattern that loads API credentials and environment variables from a dotenv file for VLM evaluation pipelines.

Description

VLMEvalKit uses python-dotenv to load API keys (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, etc.) from a .env file located at the repository root. This is the first step in any evaluation workflow, ensuring that API credentials for judge models and API-based VLMs are available in os.environ.

The pattern works as follows:

• The .env file resides at the root of the VLMEvalKit repository.
• At startup, the framework reads this file, parses each line as a key=value pair, and injects non-empty values into the process environment.
• This makes credentials accessible to all downstream modules — including API-based VLM wrappers (GPT-4o, Claude, Gemini) and judge LLMs used for evaluation scoring.
• The pattern also configures Python logging with a timestamped format (%(asctime)s %(levelname)s %(message)s), establishing consistent log output across all evaluation runs.

By centralizing credential management in a single .env file, VLMEvalKit avoids hardcoding secrets in source code and provides a single location for users to configure all provider API keys.

Usage

Use this principle at the start of any VLM evaluation workflow. It is required before:

• Running inference with API-based models (GPT-4o, Claude, Gemini, etc.)
• Running evaluation with judge LLMs that require API access
• Executing the main evaluation pipeline via run.py

The .env file should contain API keys for all providers you plan to use. For example:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AIza...

Theoretical Basis

The dotenv pattern is rooted in the 12-Factor App methodology, specifically Factor III: Store config in the environment. The core principles are:

• Separation of secrets from code — API keys and credentials are never committed to version control. They live in a .env file that is listed in .gitignore.
• Environment-based configuration — The application reads configuration from environment variables, making it portable across development, CI, and production environments.
• Single source of truth — All credentials are managed in one file, avoiding duplication and inconsistency.

The pseudocode for this pattern is:

1. Locate .env file at repository root
2. Open and read the file line by line
3. For each line:
   a. Skip comments (lines starting with #) and empty lines
   b. Parse as key=value pair
   c. If value is non-empty:
      - Set os.environ[key] = value
4. Configure Python logging with timestamped format
5. Environment is now ready for API calls

This approach ensures that:

• Secrets are never exposed in source code or logs
• Different users and CI environments can provide their own credentials without modifying code
• The loading step is idempotent — calling it multiple times does not cause errors

Related Pages

• Implementation:Open_compass_VLMEvalKit_Load_Env