Implementation:BerriAI Litellm Search API Router

Attribute	Value
Sources	litellm/router_utils/search_api_router.py
Domains	Router, Utilities, Search API, Load Balancing, Fallbacks
last_updated	2026-02-15 16:00 GMT

Overview

The Search API Router is a static utility class that handles routing, load balancing, and fallback logic for search API calls through the LiteLLM Router.

Description

This module provides the SearchAPIRouter class with static methods for integrating search tools into the LiteLLM Router's retry and fallback infrastructure. It supports: (1) syncing search tool configurations from the database to the router via update_router_search_tools; (2) finding search tools by name with get_matching_search_tools; (3) executing search requests with automatic retries and fallbacks using async_search_with_fallbacks, which leverages the router's existing async_function_with_fallbacks mechanism; and (4) a helper function async_search_with_fallbacks_helper that performs random load balancing across matching search tool providers and invokes the underlying litellm.asearch function with the selected provider's configuration (search_provider, api_key, api_base).

Usage

Import SearchAPIRouter when integrating search functionality into the LiteLLM Router. The router calls these methods internally when handling search requests.

Code Reference

Source Location

litellm/router_utils/search_api_router.py

Class: SearchAPIRouter

class SearchAPIRouter:
    """Static utility class for routing search API calls through the LiteLLM router."""

Key Methods

Method	Signature	Description
`update_router_search_tools`	`@staticmethod async def update_router_search_tools(router_instance: Any, search_tools: list)`	Syncs search tools from the database to the router instance
`get_matching_search_tools`	`@staticmethod def get_matching_search_tools(router_instance: Any, search_tool_name: str) -> list`	Returns all search tools matching the given name; raises `ValueError` if none found
`async_search_with_fallbacks`	`@staticmethod async def async_search_with_fallbacks(router_instance: Any, original_function: Callable, **kwargs)`	Executes a search API call with the router's retry/fallback infrastructure
`async_search_with_fallbacks_helper`	`@staticmethod async def async_search_with_fallbacks_helper(router_instance: Any, model: str, original_generic_function: Callable, **kwargs)`	Selects a search tool via random choice and invokes the original search function

Import

from litellm.router_utils.search_api_router import SearchAPIRouter

I/O Contract

Inputs (async_search_with_fallbacks)

Parameter	Type	Description
`router_instance`	`Any`	The LiteLLM Router instance with search_tools and fallback infrastructure
`original_function`	`Callable`	The original `litellm.asearch` function to call
`**kwargs`	`dict`	Search parameters including `search_tool_name` or `model`, `query`, etc.

Outputs (async_search_with_fallbacks)

Return Type	Description
`SearchResponse`	The response from the search API provider
Raises `ValueError`	If `search_tool_name`/`model` is not provided, or search tool/provider not found

Usage Examples

from litellm.router_utils.search_api_router import SearchAPIRouter

# Update router with search tools from the database
await SearchAPIRouter.update_router_search_tools(
    router_instance=router,
    search_tools=[
        {
            "search_tool_id": "tool-1",
            "search_tool_name": "web-search",
            "litellm_params": {
                "search_provider": "tavily",
                "api_key": "tvly-...",
            },
            "search_tool_info": {},
        },
    ],
)

# Execute a search with fallbacks
import litellm
response = await SearchAPIRouter.async_search_with_fallbacks(
    router_instance=router,
    original_function=litellm.asearch,
    search_tool_name="web-search",
    query="latest news about AI",
)

Related Pages

BerriAI_Litellm_Simple_Shuffle_Strategy - Random selection strategy used for search tool load balancing
BerriAI_Litellm_Pattern_Match_Deployments - Pattern-based deployment routing (complementary routing mechanism)

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment