Implementation:BerriAI Litellm Argilla Logger

Attribute	Value
Sources	`litellm/integrations/argilla.py`
Domains	Logging, Observability, Data Annotation, Integrations
Last Updated	2026-02-15 16:00 GMT

Overview

The ArgillaLogger is a batch logging integration that sends LLM completion data to Argilla for human annotation and feedback workflows.

Description

ArgillaLogger extends CustomBatchLogger to collect LLM request/response payloads and push them to an Argilla dataset via the Argilla REST API. It supports configurable field transformation mappings (via litellm.argilla_transformation_object), sampling rates, and batched uploads. The logger accumulates events in an in-memory queue and flushes them in batches to Argilla's /api/v1/datasets/{dataset}/records/bulk endpoint. It also integrates with LiteLLM's async_dataset_hook system, allowing custom loggers to filter or modify data before it is sent.

Usage

Import and register ArgillaLogger when you want to send LLM completions to an Argilla instance for annotation. Requires setting litellm.argilla_transformation_object to define the field mapping, plus providing an Argilla API key, base URL, and dataset name (via constructor arguments or environment variables).

Code Reference

Source Location

litellm/integrations/argilla.py

Signature

class ArgillaLogger(CustomBatchLogger):
    def __init__(
        self,
        argilla_api_key: Optional[str] = None,
        argilla_dataset_name: Optional[str] = None,
        argilla_base_url: Optional[str] = None,
        **kwargs,
    )

Import

from litellm.integrations.argilla import ArgillaLogger

I/O Contract

Inputs

Parameter	Type	Required	Description
`argilla_api_key`	`Optional[str]`	No	Argilla API key. Falls back to `ARGILLA_API_KEY` env var.
`argilla_dataset_name`	`Optional[str]`	No	Argilla dataset name. Falls back to `ARGILLA_DATASET_NAME` env var or `"litellm-completion"`.
`argilla_base_url`	`Optional[str]`	No	Argilla instance base URL. Falls back to `ARGILLA_BASE_URL` env var or `"http://localhost:6900/"`.
`kwargs`	`dict`	No	Additional keyword arguments passed to `CustomBatchLogger`.

Key Methods

Method	Returns	Description
`log_success_event(kwargs, response_obj, start_time, end_time)`	`None`	Synchronous success event logging with sampling and batching.
`async_log_success_event(kwargs, response_obj, start_time, end_time)`	`None`	Asynchronous success event logging with dataset hook support.
`async_log_failure_event(kwargs, response_obj, start_time, end_time)`	`None`	Asynchronous failure event logging.
`async_send_batch()`	`None`	Sends accumulated items to Argilla via PUT to the bulk records endpoint.

Outputs

Output	Type	Description
Side effect	HTTP request	Sends `ArgillaItem` records to the Argilla dataset via REST API.

Usage Examples

import litellm

# Define how StandardLoggingPayload fields map to Argilla dataset fields
litellm.argilla_transformation_object = {
    "chat_input": "messages",
    "chat_output": "response",
    "model_name": "model",
}

# Register the callback
litellm.success_callback = ["argilla"]

# Direct instantiation
from litellm.integrations.argilla import ArgillaLogger

logger = ArgillaLogger(
    argilla_api_key="my-api-key",
    argilla_dataset_name="llm-annotations",
    argilla_base_url="https://argilla.example.com",
)

Related Pages

BerriAI_Litellm_Literal_AI_Logger - another batch logging integration
BerriAI_Litellm_Braintrust_Logger - another observability integration
BerriAI_Litellm_S3_V2_Logger - another batch logging integration for storage

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment