Environment:Microsoft BIPIA OpenAI API Environment
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, LLM_Security |
| Last Updated | 2026-02-14 15:00 GMT |
Overview
OpenAI API environment with `openai==0.28.1` (legacy SDK) for GPT-3.5/GPT-4 inference and GPT-based model evaluation of attack success rates.
Description
This environment provides the OpenAI API client required for two distinct purposes in the BIPIA benchmark: (1) running GPT-3.5 and GPT-4 as target LLMs for evaluating their robustness to indirect prompt injection attacks, and (2) using GPT models as evaluators to judge whether an attack was successful (model-based ASR evaluation). The codebase uses the legacy `openai==0.28.1` SDK with `openai.ChatCompletion.create` and `openai.Completion.create` interfaces. API credentials are configured via YAML config files, not environment variables.
Usage
Use this environment when running GPT-3.5/GPT-4 inference for benchmark evaluation, model-based ASR evaluation using GPT as a judge, or black-box defense experiments that use GPT models. This is the mandatory prerequisite for the GPTModel wrappers and ModelEval evaluator. No GPU hardware is required for API-based models.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Any (Linux recommended) | API calls do not require specific OS |
| Hardware | No GPU required | All computation is done server-side by OpenAI |
| Network | Internet access | Required for OpenAI API calls |
| Python | >= 3.8 | Required by the bipia package |
Dependencies
Python Packages
- `openai` == 0.28.1 (pinned exact version; uses legacy ChatCompletion/Completion API)
Credentials
The following credentials must be set in YAML config files (e.g., `config/gpt35.yaml`, `config/gpt4.yaml`):
- `api_key`: OpenAI API key for authentication.
- `api_base`: (Optional) Custom API base URL for Azure OpenAI or proxy endpoints.
- `api_type`: (Optional) API type, e.g., `azure` for Azure OpenAI Service.
- `api_version`: (Optional) API version string for Azure OpenAI.
- `engine`: (Optional) Azure deployment engine name.
- `model`: Model name (e.g., `gpt-3.5-turbo`, `gpt-4`).
WARNING: Never commit YAML config files containing actual API keys to version control.
Quick Install
# Install pinned OpenAI SDK version
pip install openai==0.28.1
Code Evidence
API key passed via config from `bipia/model/gpt.py:51-56`:
response = openai.ChatCompletion.create(
api_key=self.config.get("api_key", None),
api_base=self.config.get("api_base", None),
api_type=self.config.get("api_type", None),
api_version=self.config.get("api_version", None),
engine=self.config.get("engine", None),
GPT config YAML structure from `config/gpt35.yaml`:
llm_name: gpt35
model: gpt-3.5-turbo
chat: true
Pinned openai version from `pyproject.toml:34`:
"openai==0.28.1",
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `openai.error.RateLimitError` | API rate limit exceeded | The code auto-retries after parsing the delay from the error message |
| `openai.error.InvalidRequestError` | Prompt too long or invalid parameters | The code catches this and returns empty results; reduce prompt length |
| `openai.error.APIConnectionError` | Network connectivity issue | Check internet connection; the code auto-retries after 1 second |
| `ImportError: cannot import name 'ChatCompletion' from 'openai'` | Wrong openai version installed | `pip install openai==0.28.1` (must use legacy SDK) |
Compatibility Notes
- Legacy SDK only: The codebase uses `openai==0.28.1` which is the legacy SDK. The newer `openai>=1.0` SDK has a completely different API interface and is not compatible.
- Azure OpenAI: Supported via `api_type`, `api_base`, `api_version`, and `engine` config fields.
- No GPU needed: API-based models run entirely server-side; only a network connection is required.