Implementation:Bentoml BentoML Request Context

Knowledge Sources	Bentoml_BentoML
Domains	HTTP Context, Request Handling, Service Lifecycle
Last Updated	2026-02-13 15:00 GMT

Overview

Provides request-scoped and service-scoped context objects for managing HTTP request/response metadata, trace information, and component state within BentoML services.

Description

This module defines several context classes that together form the runtime context system for BentoML services. The ServiceContext class manages per-request state using Python contextvars, including access to the current Starlette request and a mutable ResponseContext for setting headers, cookies, status codes, and background tasks. The Metadata abstract class provides a mutable mapping interface for HTTP headers. The _ServiceTraceContext class exposes OpenTelemetry trace and span identifiers, as well as a request-unique request_id. The _ComponentContext class (aliased as server_context and component_context) holds bento deployment metadata such as bento name, version, service type, worker index, and Yatai deployment details. The module also provides a request_temp_dir() helper that creates a request-unique temporary directory, automatically cleaned up when the request scope exits.

Usage

Use ServiceContext as a context manager (via in_request) during HTTP request handling to establish request and response context variables. Access trace_context for distributed tracing information and server_context / component_context for service-level metadata. Use the ResponseContext to set custom response headers, cookies, status codes, and background tasks from within API handlers.

Code Reference

Source Location

Repository: Bentoml_BentoML
File: src/bentoml/_internal/context.py
Lines: 1-252

Signature

class Metadata(t.Mapping[str, str], ABC):
    def __setitem__(self, key: str, value: str) -> None: ...
    def __delitem__(self, key: str) -> None: ...
    def __ior__(self, other: t.Mapping[t.Any, t.Any]) -> Metadata: ...
    def __or__(self, other: t.Mapping[t.Any, t.Any]) -> Metadata: ...
    def update(self, other: t.Mapping[t.Any, t.Any]) -> None: ...
    def setdefault(self, key: str, value: str) -> str: ...
    def append(self, key: str, value: str) -> None: ...
    def mutablecopy(self) -> Metadata: ...

class ServiceContext:
    def in_request(self, request: starlette.requests.Request) -> t.Generator[ServiceContext, None, None]: ...
    @property
    def request(self) -> starlette.requests.Request: ...
    @property
    def response(self) -> ResponseContext: ...
    @property
    def temp_dir(self) -> str: ...

    class ResponseContext:
        metadata: Metadata
        cookies: list[Cookie]
        status_code: int
        background: BackgroundTasks
        def set_cookie(self, key: str, value: str, ...) -> None: ...

class _ServiceTraceContext:
    @property
    def trace_id(self) -> int: ...
    @property
    def sampled(self) -> int: ...
    @property
    def span_id(self) -> int: ...
    @property
    def request_id(self) -> t.Optional[int]: ...
    @property
    def service_name(self) -> t.Optional[str]: ...

class _ComponentContext:
    bento_name: str
    bento_version: str
    service_type: str | None
    service_name: str | None
    worker_index: int | None
    service_routes: list[str]

Import

from bentoml._internal.context import ServiceContext
from bentoml._internal.context import trace_context
from bentoml._internal.context import server_context, component_context
from bentoml._internal.context import request_temp_dir

I/O Contract

Inputs

Name	Type	Required	Description
request	starlette.requests.Request	Yes	The incoming HTTP request object passed to in_request context manager

Outputs

Name	Type	Description
ServiceContext	ServiceContext	Yields the service context with request/response accessors
trace_id	int	OpenTelemetry trace identifier from the current span
request_id	Optional[int]	Unique identifier for the inbound request
temp_dir	str	Path to request-scoped temporary directory

Usage Examples

# Accessing request context within a BentoML service handler
from bentoml._internal.context import ServiceContext, trace_context, server_context

ctx = ServiceContext()

# Using the context manager during request handling
with ctx.in_request(starlette_request) as svc_ctx:
    # Access the current request
    request = svc_ctx.request

    # Set a custom response header
    svc_ctx.response.metadata["X-Custom-Header"] = "value"

    # Set a cookie on the response
    svc_ctx.response.set_cookie("session_id", "abc123", httponly=True)

    # Change the response status code
    svc_ctx.response.status_code = 201

    # Get a request-unique temporary directory
    tmp = svc_ctx.temp_dir

# Accessing trace context
print(f"Trace ID: {trace_context.trace_id}")
print(f"Span ID: {trace_context.span_id}")
print(f"Request ID: {trace_context.request_id}")

# Accessing component/server context
print(f"Bento: {server_context.bento_name}:{server_context.bento_version}")
print(f"Worker: {server_context.worker_index}")

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment