Implementation:Explodinggradients Ragas CostCallbackHandler Class

Knowledge Sources	Explodinggradients_Ragas
Domains	Cost_Tracking, Observability
Last Updated	2026-02-10 00:00 GMT

Overview

Token usage tracking and cost computation callback for LLM API calls during Ragas evaluations, supporting OpenAI, Anthropic, Bedrock, and Azure providers.

Description

TokenUsage is a Pydantic model tracking input/output tokens and model name. CostCallbackHandler is a LangChain callback handler that collects token usage from LLM calls and computes total cost. Provider-specific parsers (get_token_usage_for_openai, get_token_usage_for_anthropic, get_token_usage_for_bedrock, get_token_usage_for_azure_ai) extract usage data from different LLM responses.

Usage

Use CostCallbackHandler with a provider-specific parser when you need to track and calculate the cost of LLM calls during evaluation.

Code Reference

Source Location

• Repository: Explodinggradients_Ragas
• File: src/ragas/cost.py
• Lines: 15-234

Signature

class TokenUsage(BaseModel):
    input_tokens: int
    output_tokens: int
    model: str = ""
    def cost(self, cost_per_input_token: float, cost_per_output_token: Optional[float] = None) -> float:
        ...

class CostCallbackHandler(BaseCallbackHandler):
    def __init__(self, token_usage_parser: TokenUsageParser) -> None:
        ...
    def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
        ...
    def total_cost(self, cost_per_input_token=None, cost_per_output_token=None, per_model_costs=None) -> float:
        ...
    def total_tokens(self) -> Union[TokenUsage, List[TokenUsage]]:
        ...

def get_token_usage_for_openai(llm_result) -> TokenUsage:
    ...
def get_token_usage_for_anthropic(llm_result) -> TokenUsage:
    ...
def get_token_usage_for_bedrock(llm_result) -> TokenUsage:
    ...
def get_token_usage_for_azure_ai(llm_result) -> TokenUsage:
    ...

Import

from ragas.cost import CostCallbackHandler, TokenUsage, get_token_usage_for_openai

I/O Contract

Inputs

Name	Type	Required	Description
token_usage_parser	Callable	Yes	Parser function for the LLM provider
cost_per_input_token	float	No	Cost per input token
cost_per_output_token	float	No	Cost per output token
per_model_costs	Dict[str, Tuple[float, float]]	No	Per-model cost rates

Outputs

Name	Type	Description
total_cost returns	float	Total computed cost across all LLM calls
total_tokens returns	Union[TokenUsage, List[TokenUsage]]	Aggregated token usage

Usage Examples

from ragas.cost import CostCallbackHandler, get_token_usage_for_openai

# Create cost tracker for OpenAI
cost_cb = CostCallbackHandler(token_usage_parser=get_token_usage_for_openai)

# Pass as callback during evaluation
# ... run evaluation with cost_cb in callbacks ...

# Calculate costs
total = cost_cb.total_cost(
    cost_per_input_token=0.00001,
    cost_per_output_token=0.00003,
)
print(f"Total cost: ${total:.4f}")
print(f"Total tokens: {cost_cb.total_tokens()}")