Implementation:Bentoml BentoML IO Descriptor NumpyNdarray

Knowledge Sources	Bentoml_BentoML
Domains	IO Descriptors, NumPy, API Specification
Last Updated	2026-02-13 15:00 GMT

Overview

The NumpyNdarray IO descriptor defines API specification for BentoML services that accept or return NumPy ndarray data, supporting dtype/shape validation, enforcement, and protobuf serialization with type-mapping between numpy dtypes and protobuf field types.

Description

The NumpyNdarray class is an IO descriptor that handles numpy.ndarray inputs and outputs for BentoML services. It extends IODescriptor["ext.NpNDArray"] and provides comprehensive array handling. Key features include:

Dtype and shape configuration: Accepts optional dtype and shape parameters. When enforce_dtype or enforce_shape is True, incoming arrays that do not match are rejected with BadInput errors. Otherwise, safe casting or reshaping is attempted.
Array validation: The validate_array method performs dtype casting via np.can_cast with "same_kind" semantics and reshaping via np.reshape, logging warnings when automatic conversions occur.
HTTP request/response handling: Parses JSON request bodies into NumPy arrays via np.array(obj, dtype=self._dtype) and serializes output arrays to JSON via obj.tolist().
gRPC protobuf serialization: Provides extensive dtype mapping infrastructure through multiple cached helper functions:
- dtypepb_to_npdtype_map: Maps protobuf dtype enums to numpy dtypes (supports float32, float64, int32, int64, uint32, uint64, bool, string).
- npdtype_to_dtypepb_map: Reverse mapping from numpy dtypes to protobuf dtype enums.
- npdtype_to_fieldpb_map / fieldpb_to_npdtype_map: Maps between numpy dtypes and protobuf field names (e.g., "float_values", "int32_values").
- Supports serialized_bytes for efficient binary transport via np.frombuffer (requires dtype to be specified).
Apache Arrow and Spark integration: Provides from_arrow and to_arrow methods for RecordBatch conversion, and spark_schema for PySpark StructType generation for batch inference.
OpenAPI schema generation: Produces array schemas with item types inferred from the configured numpy dtype (integer for int-like, number for float/complex).

The module also defines the helper function _is_matched_shape which compares shapes while treating -1 as a wildcard dimension.

Usage

Use this descriptor for BentoML services that process numerical array data, such as scikit-learn or TensorFlow model inference endpoints. It is specified as the input or output parameter in the @svc.api decorator.

Code Reference

Source Location

Repository: Bentoml_BentoML
File: src/bentoml/_internal/io_descriptors/numpy.py
Lines: 1-670

Signature

class NumpyNdarray(
    IODescriptor["ext.NpNDArray"],
    descriptor_id="bentoml.io.NumpyNdarray",
    proto_fields=("ndarray",),
):
    def __init__(
        self,
        dtype: str | ext.NpDTypeLike | None = None,
        enforce_dtype: bool = False,
        shape: tuple[int, ...] | None = None,
        enforce_shape: bool = False,
    ): ...

    def input_type(self) -> LazyType[ext.NpNDArray]: ...
    def to_spec(self) -> dict[str, t.Any]: ...
    @classmethod
    def from_spec(cls, spec: dict[str, t.Any]) -> t.Self: ...
    def openapi_schema(self) -> Schema: ...
    def validate_array(self, arr: ext.NpNDArray, exception_cls: t.Type[Exception] = BadInput) -> ext.NpNDArray: ...
    async def from_http_request(self, request: Request) -> ext.NpNDArray: ...
    async def to_http_response(self, obj: ext.NpNDArray, ctx: Context | None = None): ...
    def _from_sample(self, sample: ext.NpNDArray | t.Sequence[t.Any]) -> ext.NpNDArray: ...
    async def from_proto(self, field: pb.NDArray | bytes) -> ext.NpNDArray: ...
    async def to_proto(self, obj: ext.NpNDArray) -> pb.NDArray: ...
    def from_arrow(self, batch: pyarrow.RecordBatch) -> ext.NpNDArray: ...
    def to_arrow(self, arr: ext.NpNDArray) -> pyarrow.RecordBatch: ...
    def spark_schema(self) -> pyspark.sql.types.StructType: ...

# Module-level helper functions
def dtypepb_to_npdtype_map(version: str = LATEST_PROTOCOL_VERSION) -> dict[int, ext.NpDTypeLike]: ...
def npdtype_to_dtypepb_map(version: str = LATEST_PROTOCOL_VERSION) -> dict[ext.NpDTypeLike, int]: ...
def npdtype_to_fieldpb_map() -> dict[ext.NpDTypeLike, str]: ...
def fieldpb_to_npdtype_map() -> dict[str, ext.NpDTypeLike]: ...

Import

from bentoml.io import NumpyNdarray

I/O Contract

Inputs

Name	Type	Required	Description
dtype	str, np.dtype, or None	No	Target numpy dtype for the array. If provided, incoming arrays may be cast to this dtype.
enforce_dtype	bool	No	If True, raises BadInput when incoming array dtype does not match. Defaults to False.
shape	tuple[int, ...] or None	No	Expected shape for the array. Supports -1 as wildcard dimensions.
enforce_shape	bool	No	If True, raises BadInput when incoming array shape does not match. Defaults to False.

Outputs

Name	Type	Description
ext.NpNDArray	numpy.ndarray	The validated NumPy array from an incoming request, or the array to serialize for a response.

Usage Examples

from __future__ import annotations
from typing import TYPE_CHECKING, Any

import bentoml
from bentoml.io import NumpyNdarray
import numpy as np

if TYPE_CHECKING:
    from numpy.typing import NDArray

runner = bentoml.sklearn.get("sklearn_model_clf").to_runner()
svc = bentoml.legacy.Service("iris-classifier", runners=[runner])

@svc.api(input=NumpyNdarray(), output=NumpyNdarray())
def predict(input_arr: NDArray[Any]) -> NDArray[Any]:
    return runner.run(input_arr)


# Using from_sample with shape and dtype inference
input_spec = NumpyNdarray.from_sample(np.array([[1, 2, 3]]))

@svc.api(input=input_spec, output=NumpyNdarray())
async def predict_v2(input_arr: NDArray[np.int16]) -> NDArray[Any]:
    return await runner.async_run(input_arr)


# With shape enforcement
@svc.api(
    input=NumpyNdarray(shape=(2, 2), enforce_shape=True),
    output=NumpyNdarray(),
)
async def predict_strict(input_array: NDArray[Any]) -> NDArray[Any]:
    return await runner.async_run(input_array)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment