Implementation:BerriAI Litellm S3 V2 Logger

Attribute	Value
Sources	`litellm/integrations/s3_v2.py`
Domains	Logging, Cloud Storage, AWS S3, Integrations
Last Updated	2026-02-15 16:00 GMT

Overview

The S3Logger (v2) is a batch logging integration that stores LLM request/response payloads as JSON objects in an AWS S3 bucket using SigV4-signed HTTP requests.

Description

S3Logger extends both CustomBatchLogger and BaseAWSLLM to accumulate StandardLoggingPayload events in memory and periodically flush them to S3. Unlike the S3 batch API (which does not exist), each element is uploaded individually as a PUT request signed with AWS SigV4 authentication. The logger supports extensive AWS credential configuration (access keys, session tokens, profiles, IAM roles, web identity tokens, STS endpoints), configurable S3 paths, team-based and API key-based prefixes, base64 file stripping, custom endpoint URLs, SSL verification control, and both sync and async upload paths. It also provides a download method for retrieving stored objects from cold storage.

Usage

Import and register S3Logger when you need to persist LLM logs to S3 for compliance, auditing, or cold storage. Configure via litellm.s3_callback_params or constructor arguments.

Code Reference

Source Location

litellm/integrations/s3_v2.py

Signature

class S3Logger(CustomBatchLogger, BaseAWSLLM):
    def __init__(
        self,
        s3_bucket_name: Optional[str] = None,
        s3_path: Optional[str] = None,
        s3_region_name: Optional[str] = None,
        s3_api_version: Optional[str] = None,
        s3_use_ssl: bool = True,
        s3_verify: Optional[bool] = None,
        s3_endpoint_url: Optional[str] = None,
        s3_aws_access_key_id: Optional[str] = None,
        s3_aws_secret_access_key: Optional[str] = None,
        s3_aws_session_token: Optional[str] = None,
        s3_aws_session_name: Optional[str] = None,
        s3_aws_profile_name: Optional[str] = None,
        s3_aws_role_name: Optional[str] = None,
        s3_aws_web_identity_token: Optional[str] = None,
        s3_aws_sts_endpoint: Optional[str] = None,
        s3_flush_interval: Optional[int] = DEFAULT_S3_FLUSH_INTERVAL_SECONDS,
        s3_batch_size: Optional[int] = DEFAULT_S3_BATCH_SIZE,
        s3_config=None,
        s3_use_team_prefix: bool = False,
        s3_strip_base64_files: bool = False,
        s3_use_key_prefix: bool = False,
        **kwargs,
    )

Import

from litellm.integrations.s3_v2 import S3Logger

I/O Contract

Inputs

Parameter	Type	Required	Description
`s3_bucket_name`	`Optional[str]`	No	S3 bucket name. Can also be set via `litellm.s3_callback_params`.
`s3_region_name`	`Optional[str]`	No	AWS region name.
`s3_path`	`Optional[str]`	No	S3 key path prefix.
`s3_endpoint_url`	`Optional[str]`	No	Custom S3 endpoint (for S3-compatible services).
`s3_aws_access_key_id`	`Optional[str]`	No	AWS access key ID.
`s3_aws_secret_access_key`	`Optional[str]`	No	AWS secret access key.
`s3_use_team_prefix`	`bool`	No	If True, prefix object keys with team alias. Default `False`.
`s3_use_key_prefix`	`bool`	No	If True, prefix object keys with API key alias. Default `False`.
`s3_strip_base64_files`	`bool`	No	If True, strip base64-encoded files from payloads. Default `False`.
`s3_flush_interval`	`Optional[int]`	No	Flush interval in seconds. Defaults to `DEFAULT_S3_FLUSH_INTERVAL_SECONDS`.
`s3_batch_size`	`Optional[int]`	No	Batch size before flush. Defaults to `DEFAULT_S3_BATCH_SIZE`.

Key Methods

Method	Returns	Description
`async_log_success_event(kwargs, response_obj, start_time, end_time)`	`None`	Async success logging - queues event.
`async_log_failure_event(kwargs, response_obj, start_time, end_time)`	`None`	Async failure logging - queues event.
`async_send_batch()`	`None`	Flushes queue by creating async upload tasks for each element.
`async_upload_data_to_s3(batch_logging_element)`	`None`	Uploads a single element to S3 with SigV4 signing.
`upload_data_to_s3(batch_logging_element)`	`None`	Synchronous upload to S3.
`get_proxy_server_request_from_cold_storage_with_object_key(object_key)`	`Optional[dict]`	Downloads a stored object from S3 by key.

Outputs

Output	Type	Description
Side effect	S3 PUT	Uploads JSON payloads to S3 with SigV4-signed requests.

Usage Examples

import litellm

# Configure S3 parameters
litellm.s3_callback_params = {
    "s3_bucket_name": "my-llm-logs",
    "s3_region_name": "us-east-1",
    "s3_aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
    "s3_aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "s3_path": "logs/",
}

litellm.success_callback = ["s3"]
litellm.failure_callback = ["s3"]

# Direct instantiation
from litellm.integrations.s3_v2 import S3Logger

logger = S3Logger(
    s3_bucket_name="my-bucket",
    s3_region_name="us-west-2",
    s3_use_team_prefix=True,
)

Related Pages

BerriAI_Litellm_SQS_Logger - another AWS-based logging integration for SQS queues
BerriAI_Litellm_Argilla_Logger - another batch logging integration
BerriAI_Litellm_Literal_AI_Logger - another batch logging integration

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment