Environment:Confident ai Deepeval LLM Provider Credentials

Knowledge Sources	deepeval deepeval docs
Domains	Infrastructure, LLM_Evaluation, Security
Last Updated	2026-02-14 10:00 GMT

Overview

Credential configuration environment for LLM provider API keys and the Confident AI platform key, managed through environment variables and dotenv files.

Description

Deepeval supports 12+ LLM providers, each requiring specific API keys and configuration. The framework uses a layered configuration system: process environment variables take highest precedence, followed by dotenv files (`.env` -> `.env.<APP_ENV>` -> `.env.local`), followed by a legacy JSON keystore (`.deepeval/.deepeval`). Secret keys are wrapped in Pydantic `SecretStr` to prevent accidental logging.

Usage

Use this environment whenever running any metric evaluation that calls an LLM provider (GEval, AnswerRelevancy, Faithfulness, etc.) or when uploading results to Confident AI. At minimum, one LLM provider key and optionally the Confident AI API key are required.

System Requirements

Category	Requirement	Notes
Network	Internet access	Required for LLM API calls and Confident AI uploads
Auth	LLM provider API key	At least one provider key needed for evaluation
Auth	Confident AI API key	Optional, needed for result uploads and tracing

Dependencies

Python Packages

`python-dotenv` >= 1.1.1 - Automatic dotenv loading
`pydantic` >= 2.11.7 - SecretStr for credential wrapping
`pydantic-settings` >= 2.10.1 - Environment-based settings

Credentials

Confident AI Platform:

`CONFIDENT_API_KEY`: Primary Confident AI API key (wrapped in SecretStr)
`API_KEY`: Alias for CONFIDENT_API_KEY
`CONFIDENT_BASE_URL`: Custom API endpoint URL (optional)
`CONFIDENT_REGION`: Region hint (e.g., `EU` prefix auto-routes to EU endpoint)

OpenAI:

`OPENAI_API_KEY`: OpenAI API key
`OPENAI_MODEL_NAME`: Model identifier (e.g., `gpt-4`)
`OPENAI_COST_PER_INPUT_TOKEN`: Optional cost tracking
`OPENAI_COST_PER_OUTPUT_TOKEN`: Optional cost tracking

Anthropic:

`ANTHROPIC_API_KEY`: Anthropic API key
`ANTHROPIC_MODEL_NAME`: Model identifier (e.g., `claude-3-sonnet`)

AWS Bedrock:

`AWS_ACCESS_KEY_ID`: AWS access key
`AWS_SECRET_ACCESS_KEY`: AWS secret key
`AWS_BEDROCK_MODEL_NAME`: Bedrock model identifier
`AWS_BEDROCK_REGION`: AWS region

Azure OpenAI:

`AZURE_OPENAI_API_KEY`: Azure API key
`AZURE_OPENAI_ENDPOINT`: Azure endpoint URL
`OPENAI_API_VERSION`: API version string
`AZURE_DEPLOYMENT_NAME`: Deployment name
`AZURE_MODEL_NAME`: Model name

Google/Gemini:

`GOOGLE_API_KEY`: Google API key
`GEMINI_MODEL_NAME`: Model name
`GOOGLE_GENAI_USE_VERTEXAI`: Enable Vertex AI mode
`GOOGLE_CLOUD_PROJECT`: GCP project ID (for Vertex AI)
`GOOGLE_CLOUD_LOCATION`: GCP location (for Vertex AI)
`GOOGLE_SERVICE_ACCOUNT_KEY`: Service account key path

Other Providers:

`DEEPSEEK_API_KEY`: DeepSeek API key
`GROK_API_KEY`: Grok API key
`LITELLM_API_KEY`: LiteLLM API key
`MOONSHOT_API_KEY`: Kimi/Moonshot API key
`OPENROUTER_API_KEY`: OpenRouter API key
`PORTKEY_API_KEY`: Portkey API key
`LOCAL_MODEL_API_KEY`: Local model API key
`LOCAL_MODEL_BASE_URL`: Local model endpoint

Quick Install

# Set up credentials via environment
export OPENAI_API_KEY="sk-..."
export CONFIDENT_API_KEY="..."

# Or use a .env file (recommended)
echo 'OPENAI_API_KEY=sk-...' >> .env
echo 'CONFIDENT_API_KEY=...' >> .env

# Or use the CLI
deepeval login   # Interactive Confident AI login
deepeval set --provider openai --api-key sk-...

Code Evidence

Secret key protection from `deepeval/key_handler.py:199-206`:

# hard stop on secrets: never write to disk
if _is_secret_key(key):
    logger.warning(
        "%s is a secret setting, refusing to persist. "
        "Keep your secrets in .env or .env.local instead.",
        _env_key_for_legacy_enum(key),
    )
    return

Legacy keyfile deprecation warning from `deepeval/key_handler.py:252-258`:

logger.warning(
    "Reading secret '%s' from legacy %s/%s. Persisting API keys in plaintext is deprecated. "
    "Move this to your environment (.env / .env.local). This fallback will be removed in a future release.",
    _env_key_for_legacy_enum(key),
    HIDDEN_DIR,
    KEY_FILE,
)

Required parameter validation from `deepeval/utils.py:886-916`:

def require_param(param=None, *, provider_label, env_var_name, param_hint):
    if param is None:
        raise DeepEvalError(
            f"{provider_label} is missing a required parameter. "
            f"Set {env_var_name} in your environment or pass "
            f"{param_hint}."
        )
    return param

EU region auto-detection (API key prefix routing):

# API keys with "confident_eu_" prefix auto-route to EU endpoint

Common Errors

Error Message	Cause	Solution
`{Provider} is missing a required parameter. Set {ENV_VAR} in your environment or pass {param}.`	API key not configured for the selected provider	Set the required environment variable or pass the key as a constructor argument
`Reading secret '...' from legacy .deepeval/.deepeval. Persisting API keys in plaintext is deprecated.`	API key stored in legacy JSON keyfile	Move secret to `.env` or `.env.local` file
`{key} is a secret setting, refusing to persist.`	Attempted to write a secret via the CLI settings manager	Use `.env` file or environment variable instead
`Oh no! Please provide an api key string to login.`	Empty or invalid API key passed to `deepeval.login()`	Provide a valid Confident AI API key string

Compatibility Notes

Dotenv Loading Order: `.env` -> `.env.<APP_ENV>` -> `.env.local`. Later files do NOT override earlier values. Set `DEEPEVAL_DISABLE_DOTENV=1` to skip all dotenv loading.
Legacy Keyfile: The `.deepeval/.deepeval` JSON keystore is deprecated for secrets. Set `DEEPEVAL_DISABLE_LEGACY_KEYFILE=1` to skip legacy keyfile loading entirely.
SecretStr Wrapping: All API keys use Pydantic `SecretStr`, preventing accidental exposure in logs. Use `.get_secret_value()` to access the raw string.
Provider Selection: Use `USE_<PROVIDER>_MODEL=true` environment variables (e.g., `USE_ANTHROPIC_MODEL=true`) or pass the model class directly in code to select a provider.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment