Implementation:Bentoml BentoML Monitor API

Knowledge Sources	Bentoml_BentoML
Domains	Monitoring, Observability, Data Logging, Dependency Injection
Last Updated	2026-02-13 15:00 GMT

Overview

Provides the public monitor() context manager API for logging inference data (features, predictions, and targets) to configurable monitoring backends in BentoML services.

Description

This module exposes the monitor() function, which is the primary public API for BentoML's monitoring system. It is a context manager decorated with @inject from simple_di that yields a MonitorBase instance configured from BentoML's container settings.

The function maintains a module-level cache (_MONITOR_INSTANCES) that maps monitor names to their instances, ensuring each named monitor is created only once. On first use for a given name, the monitor class is resolved from the monitor_class parameter:

None or "default" resolves to DefaultMonitor (file-based logging)
"otlp" resolves to OTLPMonitor (OpenTelemetry Protocol)
A string like "mypackage.MyMonitor" resolves to a custom class via LazyType
A class object is used directly
If monitoring is disabled in configuration, NoOpMonitor is used

The context manager calls start_record() on entry and stop_record() on exit, bracketing the data logging session. Within the context, the yielded monitor instance provides log(), log_batch(), and log_table() methods for recording data.

The function has multiple @t.overload signatures to provide precise type hints depending on the monitor_class argument type.

Usage

Use bentoml.monitor("name") as a context manager within service API handlers to log input features, model predictions, and ground truth targets. Configure the monitoring backend via BentoML's configuration file or environment variables.

Code Reference

Source Location

Repository: Bentoml_BentoML
File: src/bentoml/_internal/monitoring/api.py
Lines: 1-136

Signature

@contextlib.contextmanager
@inject
def monitor(
    name: str,
    monitor_class: type[MT] | str | None = Provide[
        BentoMLContainer.config.monitoring.type
    ],
    monitor_options: dict[str, t.Any] | None = Provide[
        BentoMLContainer.config.monitoring.options
    ],
) -> t.Generator[MT | MonitorBase[t.Any], None, None]: ...

Import

import bentoml

# Via the public API
with bentoml.monitor("my_monitor") as m:
    m.log(value, "column_name", "feature", "numerical")

# Direct import
from bentoml._internal.monitoring.api import monitor

I/O Contract

Inputs

Name	Type	Required	Description
name	str	Yes	Unique name identifying this monitor instance; cached globally
monitor_class	type, str, or None	No (injected from config)	Monitor backend class: "default", "otlp", a fully-qualified class name string, or a class object
monitor_options	dict[str, Any] or None	No (injected from config)	Additional keyword arguments passed to the monitor constructor

Outputs

Name	Type	Description
MonitorBase instance	MonitorBase[Any]	The monitor instance yielded by the context manager, providing log(), log_batch(), and log_table() methods

Usage Examples

import bentoml

# Basic monitoring with default backend
@svc.api(input=NumpyNdarray(), output=NumpyNdarray())
def predict(input_data):
    result = runner.run(input_data)

    with bentoml.monitor("inference_monitor") as m:
        m.log(input_data[0], "feature_x", "feature", "numerical")
        m.log(input_data[1], "feature_y", "feature", "numerical")
        m.log(result, "prediction", "prediction", "numerical")

    return result

# Batch logging
with bentoml.monitor("batch_monitor") as m:
    m.log_batch([1, 2, 3], "x", "feature", "numerical")
    m.log_batch([4, 5, 6], "y", "feature", "numerical")
    m.log_batch([7, 8, 9], "prediction", "prediction", "numerical")

# Using OTLP backend
with bentoml.monitor("otlp_monitor", monitor_class="otlp") as m:
    m.log(42, "score", "prediction", "numerical")

# Using a custom monitor class
with bentoml.monitor(
    "custom_monitor",
    monitor_class="mypackage.monitors.CustomMonitor",
    monitor_options={"endpoint": "http://localhost:9090"},
) as m:
    m.log("cat", "category", "feature", "categorical")

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment