Implementation:BerriAI Litellm Lowest Cost Strategy

Attribute	Value
Sources	litellm/router_strategy/lowest_cost.py
Domains	Router, Strategy, Cost Optimization
last_updated	2026-02-15 16:00 GMT

Overview

The Lowest Cost Strategy is a router deployment selection strategy that picks the LLM deployment with the lowest per-token cost from a set of healthy deployments, while respecting TPM and RPM limits.

Description

This module provides the LowestCostLoggingHandler class, which extends CustomLogger to track per-deployment token usage and select the cheapest available deployment for each request. On each successful API call, it logs token usage (TPM) and request counts (RPM) into a DualCache keyed by model group, deployment ID, and the current minute. When selecting a deployment, it computes the combined input and output cost per token for each candidate (from litellm.model_cost or from litellm_params), filters out those exceeding their TPM/RPM limits, sorts by cost, and returns the cheapest option.

Usage

Import and instantiate this class when configuring the LiteLLM Router with routing_strategy="cost-based-routing". The router automatically uses it to log usage events and select deployments.

Code Reference

Source Location

litellm/router_strategy/lowest_cost.py

Class: LowestCostLoggingHandler

class LowestCostLoggingHandler(CustomLogger):
    test_flag: bool = False
    logged_success: int = 0
    logged_failure: int = 0

    def __init__(self, router_cache: DualCache, routing_args: dict = {}):

Key Methods

Method	Signature	Description
`log_success_event`	`def log_success_event(self, kwargs, response_obj, start_time, end_time)`	Sync callback that updates TPM/RPM counters in cache after a successful call
`async_log_success_event`	`async def async_log_success_event(self, kwargs, response_obj, start_time, end_time)`	Async callback that updates TPM/RPM counters in cache after a successful call
`async_get_available_deployments`	`async def async_get_available_deployments(self, model_group: str, healthy_deployments: list, messages: Optional[List[Dict[str, str]]] = None, input: Optional[Union[str, List]] = None, request_kwargs: Optional[Dict] = None)`	Selects the deployment with the lowest cost that is within TPM/RPM limits

Import

from litellm.router_strategy.lowest_cost import LowestCostLoggingHandler

I/O Contract

Inputs

Parameter	Type	Description
`router_cache`	`DualCache`	Shared cache instance for storing usage counters
`routing_args`	`dict`	Optional routing configuration arguments
`model_group`	`str`	The model group name to select a deployment for
`healthy_deployments`	`list`	List of deployment dictionaries currently considered healthy
`messages`	`Optional[List[Dict[str, str]]]`	Messages to estimate input token count
`input`	`Optional[Union[str, List]]`	Text input to estimate input token count

Outputs

Return Type	Description
`Optional[dict]`	The selected deployment dictionary with the lowest cost, or `None` if no deployments are available within limits

Usage Examples

from litellm.caching.caching import DualCache
from litellm.router_strategy.lowest_cost import LowestCostLoggingHandler

# Initialize with a shared router cache
cache = DualCache()
handler = LowestCostLoggingHandler(router_cache=cache)

# The handler is typically registered as a callback by the Router
# and called internally during deployment selection:
deployment = await handler.async_get_available_deployments(
    model_group="gpt-4",
    healthy_deployments=[
        {
            "model_info": {"id": "deployment-1"},
            "litellm_params": {
                "model": "gpt-4",
                "input_cost_per_token": 0.00003,
                "output_cost_per_token": 0.00006,
            },
            "tpm": 100000,
            "rpm": 500,
        },
        {
            "model_info": {"id": "deployment-2"},
            "litellm_params": {
                "model": "gpt-4-turbo",
                "input_cost_per_token": 0.00001,
                "output_cost_per_token": 0.00003,
            },
            "tpm": 200000,
            "rpm": 1000,
        },
    ],
    messages=[{"role": "user", "content": "Hello"}],
)

Related Pages

BerriAI_Litellm_Lowest_TPM_RPM_V2_Strategy - Alternative strategy that selects by lowest TPM/RPM usage
BerriAI_Litellm_Lowest_TPM_RPM_Strategy - Original TPM/RPM-based routing strategy
BerriAI_Litellm_Least_Busy_Strategy - Strategy that selects the least busy deployment
BerriAI_Litellm_Simple_Shuffle_Strategy - Random/weighted shuffle deployment selection

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment