Environment:Lm sys FastChat API Keys And Credentials
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Security |
| Last Updated | 2026-02-07 04:00 GMT |
Overview
Environment variables and API credentials required for FastChat's external LLM API providers, moderation services, and operational configuration.
Description
FastChat integrates with numerous external LLM API providers (OpenAI, Anthropic, Google, Mistral, etc.) for its Arena platform and MT-Bench evaluation. Each provider requires its own API key set via environment variables. Additionally, content moderation services (Azure, PhotoDNA) and operational settings (logging, heartbeat intervals, input limits) are configured through environment variables.
Usage
Use this environment when running the Chatbot Arena web server with external API providers, or when running MT-Bench evaluation with GPT-4 or Claude as judges. Also required for content moderation in production deployments.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Any | Environment variables are platform-independent |
| Network | Internet access | Required for external API calls |
Dependencies
No additional packages beyond the base FastChat installation.
Credentials
LLM API Provider Keys
- `OPENAI_API_KEY`: OpenAI API key — used in `api_provider.py`, `utils.py`, `common.py` for GPT models
- `ANTHROPIC_API_KEY`: Anthropic API key — used in `api_provider.py` for Claude models
- `GEMINI_API_KEY`: Google Gemini API key — used in `api_provider.py`
- `MISTRAL_API_KEY`: Mistral AI API key — used in `api_provider.py`
- `NVIDIA_API_KEY`: NVIDIA NIM API key — used in `api_provider.py`
- `YANDEXGPT_API_KEY`: YandexGPT API key — used in `api_provider.py`
- `REKA_API_KEY`: Reka AI API key — used in `api_provider.py`
- `AI2_API_KEY`: Allen AI API key — used in `api_provider.py`
- `BARD_API_KEY`: Google Bard API key (legacy) — used in `api_provider.py`
- `GENAI_API_KEY`: Google GenAI API key — used in `classify/label.py`
Azure and GCP Keys
- `AZURE_OPENAI_ENDPOINT`: Azure OpenAI service endpoint — used in `common.py`
- `AZURE_OPENAI_KEY`: Azure OpenAI API key — used in `common.py`
- `AZURE_IMG_MODERATION_ENDPOINT`: Azure image moderation endpoint — used in `utils.py`
- `AZURE_IMG_MODERATION_API_KEY`: Azure image moderation API key — used in `utils.py`
- `GCP_PROJECT_ID`: Google Cloud project ID — used in `api_provider.py`
- `GCP_LOCATION`: Google Cloud region — used in `api_provider.py`
Moderation Keys
- `PHOTODNA_API_KEY`: Microsoft PhotoDNA CSAM detection key — used in `utils.py`
Operational Configuration
- `LOGDIR`: Log file output directory (default: `.`) — used in `constants.py`
- `CPU_ISA`: CPU instruction set (`avx512_bf16` or `amx`) — used in `constants.py`
- `REMOTE_LOGGER_URL`: URL for remote JSON logging — used in `remote_logger.py`
- `COPILOT_ARENA_LEADERBOARD_URL`: Copilot Arena leaderboard endpoint — used in `copilot_arena.py`
- `REQUEST_TIMEOUT`: HTTP request timeout in seconds (default: 3) — used in `utils.py`
FastChat Tuning Variables
- `FASTCHAT_INPUT_CHAR_LEN_LIMIT`: Max input characters (default: 12000) — used in `constants.py`
- `FASTCHAT_BLIND_MODE_INPUT_CHAR_LEN_LIMIT`: Blind mode input limit (default: 30000) — used in `constants.py`
- `FASTCHAT_CONTROLLER_HEART_BEAT_EXPIRATION`: Controller heartbeat expiry in seconds (default: 90) — used in `constants.py`
- `FASTCHAT_WORKER_HEART_BEAT_INTERVAL`: Worker heartbeat interval in seconds (default: 45) — used in `constants.py`
- `FASTCHAT_WORKER_API_TIMEOUT`: Worker API timeout in seconds (default: 100) — used in `constants.py`
- `FASTCHAT_WORKER_API_EMBEDDING_BATCH_SIZE`: Embedding batch size (default: 4) — used in `constants.py`
Quick Install
# Set minimum required keys for MT-Bench evaluation
export OPENAI_API_KEY="your-openai-key"
# For Arena with multiple providers
export ANTHROPIC_API_KEY="your-anthropic-key"
export GEMINI_API_KEY="your-gemini-key"
# For production with moderation
export AZURE_IMG_MODERATION_ENDPOINT="your-endpoint"
export AZURE_IMG_MODERATION_API_KEY="your-key"
Code Evidence
Constants with env var defaults from `fastchat/constants.py:43-64`:
INPUT_CHAR_LEN_LIMIT = int(os.getenv("FASTCHAT_INPUT_CHAR_LEN_LIMIT", 12000))
BLIND_MODE_INPUT_CHAR_LEN_LIMIT = int(
os.getenv("FASTCHAT_BLIND_MODE_INPUT_CHAR_LEN_LIMIT", 30000)
)
CONVERSATION_TURN_LIMIT = 50
SESSION_EXPIRATION_TIME = 3600
LOGDIR = os.getenv("LOGDIR", ".")
CPU_ISA = os.getenv("CPU_ISA")
CONTROLLER_HEART_BEAT_EXPIRATION = int(
os.getenv("FASTCHAT_CONTROLLER_HEART_BEAT_EXPIRATION", 90)
)
WORKER_HEART_BEAT_INTERVAL = int(os.getenv("FASTCHAT_WORKER_HEART_BEAT_INTERVAL", 45))
WORKER_API_TIMEOUT = int(os.getenv("FASTCHAT_WORKER_API_TIMEOUT", 100))
WORKER_API_EMBEDDING_BATCH_SIZE = int(
os.getenv("FASTCHAT_WORKER_API_EMBEDDING_BATCH_SIZE", 4)
)
OpenAI key usage from `fastchat/utils.py:158`:
oai_key = os.environ.get("OPENAI_API_KEY", "")
SSL certificate env vars from `fastchat/serve/model_worker.py:421-422`:
ssl_keyfile=os.environ["SSL_KEYFILE"],
ssl_certfile=os.environ["SSL_CERTFILE"],
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `KeyError: 'OPENAI_API_KEY'` | OpenAI key not set | `export OPENAI_API_KEY="sk-..."` |
| `KeyError: 'SSL_KEYFILE'` | SSL enabled without certificates | Set `SSL_KEYFILE` and `SSL_CERTFILE` env vars, or remove `--ssl` flag |
| API timeout errors | `REQUEST_TIMEOUT` too low for slow models | `export REQUEST_TIMEOUT=30` |
Compatibility Notes
- MT-Bench vs Arena: MT-Bench only requires `OPENAI_API_KEY` (or `ANTHROPIC_API_KEY`). The Arena requires keys for all configured providers.
- Azure vs OpenAI: Azure OpenAI uses separate `AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_KEY` variables, not `OPENAI_API_KEY`.
- Heartbeat tuning: The default 90-second expiration with 45-second interval means workers can miss at most one heartbeat. For high-latency networks, increase both values proportionally.
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment