Implementation:BerriAI Litellm Simple Shuffle Strategy

Attribute	Value
Sources	litellm/router_strategy/simple_shuffle.py
Domains	Router, Strategy, Load Balancing
last_updated	2026-02-15 16:00 GMT

Overview

The Simple Shuffle Strategy returns a randomly selected deployment from the list of healthy deployments, optionally performing weighted random selection based on weight, rpm, or tpm parameters.

Description

This module provides the simple_shuffle function, which implements the simplest routing strategy in LiteLLM. It checks whether deployments have weight, rpm, or tpm values defined in their litellm_params. If any of these parameters exist, it performs a weighted random selection using random.choices with normalized weights. If no weighting parameters are present, it falls back to a uniform random selection using random.choice. The priority order for weight detection is: weight first, then rpm, then tpm.

Usage

Import this function when configuring the LiteLLM Router with routing_strategy="simple-shuffle". It is the default strategy when no specific routing strategy is configured.

Code Reference

Source Location

litellm/router_strategy/simple_shuffle.py

Function Signature

def simple_shuffle(
    llm_router_instance: LitellmRouter,
    healthy_deployments: Union[List[Any], Dict[Any, Any]],
    model: str,
) -> Dict:

Import

from litellm.router_strategy.simple_shuffle import simple_shuffle

I/O Contract

Inputs

Parameter	Type	Description
`llm_router_instance`	`LitellmRouter`	The Router instance (used for logging via `print_deployment`)
`healthy_deployments`	`Union[List[Any], Dict[Any, Any]]`	List of healthy deployment dictionaries
`model`	`str`	The model name being requested

Outputs

Return Type	Description
`Dict`	A single deployment dictionary selected either randomly or by weighted random pick

Usage Examples

from litellm.router_strategy.simple_shuffle import simple_shuffle

# With weighted deployments (by rpm)
deployment = simple_shuffle(
    llm_router_instance=router,
    healthy_deployments=[
        {"litellm_params": {"model": "gpt-4", "rpm": 100}},
        {"litellm_params": {"model": "gpt-4", "rpm": 300}},
    ],
    model="gpt-4",
)

# Without weights (uniform random)
deployment = simple_shuffle(
    llm_router_instance=router,
    healthy_deployments=[
        {"litellm_params": {"model": "gpt-4"}},
        {"litellm_params": {"model": "gpt-4"}},
    ],
    model="gpt-4",
)

Related Pages

BerriAI_Litellm_Lowest_TPM_RPM_V2_Strategy - Usage-based deployment selection
BerriAI_Litellm_Lowest_Cost_Strategy - Cost-based deployment selection
BerriAI_Litellm_Least_Busy_Strategy - Least busy deployment selection
BerriAI_Litellm_Tag_Based_Routing - Tag-based deployment filtering

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment