Heuristic:Mistralai Client python Retry Backoff Strategy

Knowledge Sources	mistralai/client-python Mistral SDK README
Domains	Reliability, API_Client
Last Updated	2026-02-15 14:00 GMT

Overview

Exponential backoff retry strategy with jitter for handling transient API errors (429, 5xx) with configurable per-request overrides.

Description

The Mistral SDK implements a robust retry mechanism using exponential backoff with random jitter. When HTTP requests fail with retryable status codes (429 Rate Limit, 500, 502, 503, 504), the SDK automatically retries with increasing delays. The backoff formula includes a random 0-1 second jitter component to prevent thundering herd problems when multiple clients hit rate limits simultaneously. The SDK also respects the server-provided `Retry-After` header when present.

Usage

Apply this heuristic when configuring the Mistral SDK for production use, especially in high-throughput scenarios or environments with unreliable network connections. Understanding the default retry behavior helps in tuning timeout and retry parameters for specific use cases.

The Insight (Rule of Thumb)

Default Values:
- Initial interval: 500ms
- Max interval: 60,000ms (60 seconds)
- Exponential base: 1.5
- Max elapsed time: 3,600,000ms (1 hour)
Retried Status Codes: 429, 500, 502, 503, 504
Jitter Formula: `sleep = (initial_interval / 1000) * exponent^retries + random(0, 1)`
Retry-After Header: SDK respects server-provided retry delays, overriding the calculated backoff
Trade-off: Aggressive retries improve reliability but increase overall request latency; 1-hour max elapsed time may be excessive for latency-sensitive applications
Per-request Override: Retry config can be overridden per API call via `RetryConfig`

Reasoning

Rate limiting (429) is the most common transient error with API services. Without jitter, synchronized retries from multiple clients create "thundering herd" effects that worsen the problem. The exponential backoff with jitter spreads retry attempts over time. The `Retry-After` header takes precedence because the server has the most accurate information about when the client should retry.

The default 1-hour max elapsed time is generous for long-running operations like fine-tuning job creation but may need to be reduced for interactive use cases.

Code Evidence

Backoff formula from `src/mistralai/client/utils/retries.py:122`:

sleep = (initial_interval / 1000) * exponent**retries + random.uniform(0, 1)
return min(sleep, max_interval / 1000)

Retry-After header parsing from `src/mistralai/client/utils/retries.py:70-93`:

def _parse_retry_after_header(response: httpx.Response) -> Optional[int]:
    """Parse Retry-After header from response.
    Returns: Retry interval in milliseconds, or None if header is missing or invalid.
    """
    retry_after_header = response.headers.get("retry-after")
    if not retry_after_header:
        return None
    try:
        seconds = float(retry_after_header)
        return round(seconds * 1000)
    except ValueError:
        pass
    # Also supports HTTP-date format

Status code matching from `src/mistralai/client/utils/retries.py:134-146`:

for code in retries.status_codes:
    if "X" in code.upper():
        code_range = int(code[0])
        status_major = res.status_code / 100
        if code_range <= status_major < code_range + 1:
            raise TemporaryError(res)
    else:
        parsed_code = int(code)
        if res.status_code == parsed_code:
            raise TemporaryError(res)

Connection error handling from `src/mistralai/client/utils/retries.py:147-156`:

except httpx.ConnectError as exception:
    if retries.config.retry_connection_errors:
        raise
    raise PermanentError(exception) from exception
except httpx.TimeoutException as exception:
    if retries.config.retry_connection_errors:
        raise
    raise PermanentError(exception) from exception

Per-request retry override example from README:

RetryConfig("backoff", BackoffStrategy(1, 50, 1.1, 100), False)
# Parameters: strategy, backoff(initial_ms, max_ms, exponent, max_elapsed_ms), retry_connection_errors

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment