Heuristic:Openai Openai python Timeout Connection Defaults
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Optimization |
| Last Updated | 2026-02-15 10:00 GMT |
Overview
The SDK uses a 10-minute total timeout (600s) with 5-second connect timeout, and connection pooling with 1000 max connections and 100 keepalive connections.
Description
The OpenAI Python SDK ships with carefully chosen default timeout and connection pool settings. The 10-minute total timeout accommodates long-running operations like image generation, audio transcription, and large file uploads. The 5-second connect timeout catches network issues quickly. Connection pooling with keepalive reduces TLS handshake overhead for repeated requests. These defaults can be overridden at the client level or per-request.
Usage
Understand these defaults when:
- Operations time out unexpectedly — the default is generous at 600s but some operations (batch processing, very large uploads) may need more.
- Tuning for low-latency — reduce timeouts for time-sensitive applications.
- High-throughput scenarios — the 1000 connection limit is high but may need adjustment for extreme concurrency.
- Per-request overrides — every API method accepts a `timeout` parameter to override the client-level default.
The Insight (Rule of Thumb)
- Action: Override timeout and connection settings on the `OpenAI()` constructor or per-request.
- Default values:
- Total timeout: `600` seconds (10 minutes)
- Connect timeout: `5.0` seconds
- Max connections: `1000`
- Max keepalive connections: `100`
- Max retries: `2`
- Trade-off: Lower timeouts improve failure detection but risk canceling slow but valid operations. Higher connection limits use more memory.
- Per-request override: Pass `timeout=httpx.Timeout(...)` to any API method for granular control.
Reasoning
The 10-minute default timeout reflects the reality that some OpenAI API operations (particularly image generation with DALL-E 3 and large audio transcriptions) can legitimately take several minutes. The 5-second connect timeout separately catches DNS resolution failures and network unreachability quickly without affecting long-running operations.
The connection pool settings (1000 max, 100 keepalive) balance memory usage with connection reuse. For most applications, these defaults provide good performance. Only high-concurrency batch processing scenarios would need to increase these values.
Code evidence from `_constants.py:8-14`:
# default timeout is 10 minutes
DEFAULT_TIMEOUT = httpx.Timeout(timeout=600, connect=5.0)
DEFAULT_MAX_RETRIES = 2
DEFAULT_CONNECTION_LIMITS = httpx.Limits(max_connections=1000, max_keepalive_connections=100)
INITIAL_RETRY_DELAY = 0.5
MAX_RETRY_DELAY = 8.0