Implementation:BerriAI Litellm Passthrough API

Property	Value
sources	litellm/passthrough/main.py
domains	Passthrough, Provider Access, Streaming, HTTP Forwarding
last_updated	2026-02-15 16:00 GMT

Overview

The Passthrough API module forwards HTTP requests directly to LLM provider endpoints, handling URL construction, authentication, request signing, streaming, and response logging without transforming the request/response payloads.

Description

This module provides a transparent proxy layer for sending raw HTTP requests to any supported LLM provider. The llm_passthrough_route/allm_passthrough_route function pair resolves the provider via get_llm_provider(), loads a BasePassthroughConfig for URL building and request signing, validates authentication headers, and dispatches the request through the provider's HTTP client. It supports both streaming and non-streaming responses. For streaming, it uses generator functions (_sync_streaming and _async_streaming) that yield raw byte chunks while collecting them for post-streaming log flushing. The module handles Bedrock application inference profile URL encoding as a special case. Request signing (e.g., for AWS Bedrock) is performed through the provider config's sign_request() method.

Usage

Import this module when you need direct, unmodified access to a provider's API endpoints, bypassing LiteLLM's standard request/response transformation. This is useful for provider-specific features not yet supported by LiteLLM's unified interfaces.

Code Reference

Source Location

Property	Value
Repository	github.com/BerriAI/litellm
File	litellm/passthrough/main.py
Lines	430
Module	litellm.passthrough.main

Signature

@client
def llm_passthrough_route(
    *,
    method: str,
    endpoint: str,
    model: str,
    custom_llm_provider: Optional[str] = None,
    api_base: Optional[str] = None,
    api_key: Optional[str] = None,
    request_query_params: Optional[dict] = None,
    request_headers: Optional[dict] = None,
    allm_passthrough_route: bool = False,
    content: Optional[Any] = None,
    data: Optional[dict] = None,
    files: Optional[RequestFiles] = None,
    json: Optional[Any] = None,
    params: Optional[QueryParamTypes] = None,
    cookies: Optional[CookieTypes] = None,
    client: Optional[Union[HTTPHandler, AsyncHTTPHandler]] = None,
    **kwargs,
) -> Union[httpx.Response, Coroutine, Generator, AsyncGenerator]

@client
async def allm_passthrough_route(
    *,
    method: str,
    endpoint: str,
    model: str,
    ...
) -> Union[httpx.Response, AsyncGenerator[Any, Any]]

Import

from litellm.passthrough.main import llm_passthrough_route, allm_passthrough_route

I/O Contract

Inputs

Parameter	Type	Required	Description
method	str	Yes	HTTP method (GET, POST, PUT, DELETE, etc.)
endpoint	str	Yes	The API endpoint path to call on the provider
model	str	Yes	Model identifier for provider resolution
custom_llm_provider	Optional[str]	No	Provider override; auto-detected from model
api_base	Optional[str]	No	Base URL override for the provider
api_key	Optional[str]	No	API key for authentication
data	Optional[dict]	No	Form data for the request
json	Optional[Any]	No	JSON body for the request
files	Optional[RequestFiles]	No	Files to upload
request_headers	Optional[dict]	No	Headers to forward from the original request

Outputs

Output	Type	Description
Non-streaming	httpx.Response	The raw HTTP response from the provider
Streaming (sync)	Generator[bytes]	Generator yielding raw byte chunks
Streaming (async)	AsyncGenerator[bytes]	Async generator yielding raw byte chunks

Usage Examples

import litellm

# Direct API call to OpenAI
response = litellm.llm_passthrough_route(
    method="POST",
    endpoint="/v1/chat/completions",
    model="openai/gpt-4",
    json={
        "model": "gpt-4",
        "messages": [{"role": "user", "content": "Hello"}],
    },
)
print(response.json())

import asyncio
import litellm

async def main():
    response = await litellm.allm_passthrough_route(
        method="GET",
        endpoint="/v1/models",
        model="openai/gpt-4",
    )
    print(response.json())

asyncio.run(main())

Related Pages

• BerriAI_Litellm_Responses_API -- The standard Responses API that transforms requests, unlike this passthrough module
• BerriAI_Litellm_Assistants_API -- Assistants API with full request/response transformation