Implementation:Bentoml BentoML Spark Batch

Knowledge Sources	Bentoml_BentoML
Domains	Batch Inference, Apache Spark
Last Updated	2026-02-13 15:00 GMT

Overview

Enables distributed batch inference by running BentoML service APIs across Apache Spark worker nodes using Arrow-based data exchange.

Description

The spark module provides the run_in_spark function, which orchestrates distributed batch inference on a Spark cluster. The workflow is:

1. Bento Distribution -- The _distribute_bento function exports the Bento to a temporary directory and adds it to Spark's distributed file cache via sparkContext.addFile, ensuring every worker node has access to the model artifact.
2. Worker-Side Loading -- The _load_bento_spark function attempts to load the Bento from the local store first; if unavailable, it falls back to importing the Bento from the SparkFiles directory (the distributed cache).
3. Processing Pipeline -- The _get_process function returns a generator-based processing function compatible with Spark's mapInArrow API. On each worker, it loads the Bento, starts a local BentoML server on a reserved free port, waits for server readiness, and creates an HTTP client. It then iterates over Arrow RecordBatches, converts input via the API's input descriptor, calls the inference endpoint, and yields output as Arrow RecordBatches.
4. Orchestration -- run_in_spark validates the API name (auto-selecting if only one API exists), distributes the Bento, creates the processing function, infers the output Spark schema from the API output descriptor (if not provided), and applies df.mapInArrow for distributed execution.

The module requires pyspark and raises MissingDependencyException if it is not installed.

Usage

Use this module to run batch inference at scale on Spark clusters, processing large DataFrames through BentoML model serving APIs with Arrow-optimized data transfer.

Code Reference

Source Location

• Repository: Bentoml_BentoML
• File: src/bentoml/_internal/batch/spark.py
• Lines: 1-171

Signature

def run_in_spark(
    bento: Bento,
    df: pyspark.sql.dataframe.DataFrame,
    spark: pyspark.sql.session.SparkSession,
    api_name: str | None = None,
    output_schema: StructType | None = None,
) -> pyspark.sql.dataframe.DataFrame: ...

Import

from bentoml._internal.batch.spark import run_in_spark
# or via public API:
import bentoml.batch
bentoml.batch.run_in_spark(bento, df, spark)

I/O Contract

Inputs

Name	Type	Required	Description
bento	Bento	Yes	The Bento object containing the inference API to run
df	pyspark.sql.DataFrame	Yes	Input Spark DataFrame to process
spark	pyspark.sql.SparkSession	Yes	The active Spark session
api_name	str or None	No	Name of the API to call; auto-detected if the Bento has only one API
output_schema	StructType or None	No	Spark output schema; inferred from API output descriptor if not provided

Outputs

Name	Type	Description
DataFrame	pyspark.sql.DataFrame	Result DataFrame with inference outputs, produced by mapInArrow

Usage Examples

import bentoml
from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()
df = spark.createDataFrame([("John", 30), ("Mike", 25)], ["name", "age"])

bento = bentoml.get("my_service:latest")
results = bentoml.batch.run_in_spark(bento, df, spark, api_name="predict")
results.show()

Related Pages

• Implementation:Bentoml_BentoML_Arrow_Conversion