Implementation:SeldonIO Seldon core Seldon Pipeline CRD Multi Modal

Field	Value
Type	Pattern Doc
Overview	Concrete pattern for composing multi-modal HuggingFace pipelines in Seldon Core 2.
Source	samples/pipelines/speech-to-sentiment.yaml:L1-22
Domains	NLP, Data_Flow
Implements Principle	SeldonIO_Seldon_core_Multi_Modal_Pipeline_Composition
External Dependencies	Kubernetes API, Kafka, custom MLServer runtimes
Knowledge Sources	Repo (https://github.com/SeldonIO/seldon-core), Doc (https://docs.seldon.io/projects/seldon-core/en/v2/)
Last Updated	2026-02-13 00:00 GMT

Code Reference

Speech-to-Sentiment Pipeline

apiVersion: mlops.seldon.io/v1alpha1
kind: Pipeline
metadata:
  name: speech-to-sentiment
spec:
  steps:
    - name: whisper
    - name: sentiment
      inputs:
      - whisper
      tensorMap:
        whisper.outputs.output: args
    - name: sentiment-input-transform
      inputs:
      - whisper
    - name: sentiment-explainer
      inputs:
      - sentiment-input-transform
  output:
    steps:
    - sentiment
    - whisper

Pipeline Step Breakdown

Step	Inputs	tensorMap	Purpose
whisper	(pipeline input)	--	Converts audio input to transcribed text
sentiment	whisper	whisper.outputs.output: args	Classifies the transcribed text as POSITIVE/NEGATIVE; remaps whisper's output tensor to sentiment's expected args input
sentiment-input-transform	whisper	--	Preprocesses whisper output for the explainer model
sentiment-explainer	sentiment-input-transform	--	Generates anchor text explanations for the sentiment classification

Key Parameters

Parameter	Example	Description
spec.steps[].name	whisper, sentiment	References a deployed Model by name
spec.steps[].inputs	["whisper"]	Upstream step dependencies; defines the pipeline DAG edges
spec.steps[].tensorMap	whisper.outputs.output: args	Remaps tensor names between steps with incompatible interfaces
spec.output.steps	["sentiment", "whisper"]	Steps whose outputs are included in the pipeline response (multi-output)

I/O Contract

Inputs

Input	Format	Description
Deployed models	Kubernetes Model resources	All referenced models must be deployed and in ModelAvailable state: whisper, sentiment, sentiment-input-transform, sentiment-explainer

Outputs

Output	Format	Description
Pipeline CRD manifest	YAML	speech-to-sentiment Pipeline with 4 steps, tensor remapping, and multi-output configuration
Pipeline response	V2 JSON	Combined outputs from both the sentiment step (label + score) and the whisper step (transcription)

Usage Examples

Deploying the pipeline

First ensure all models are deployed and available:

seldon model load -f samples/models/hf-whisper.yaml
seldon model status whisper -w ModelAvailable

seldon model load -f samples/models/hf-sentiment.yaml
seldon model status sentiment -w ModelAvailable

seldon model load -f samples/models/hf-sentiment-input-transform.yaml
seldon model status sentiment-input-transform -w ModelAvailable

seldon model load -f samples/models/hf-sentiment-explainer.yaml
seldon model status sentiment-explainer -w ModelAvailable

Then load the pipeline:

seldon pipeline load -f samples/pipelines/speech-to-sentiment.yaml
seldon pipeline status speech-to-sentiment -w PipelineReady

Sending inference to the pipeline

seldon pipeline infer speech-to-sentiment \
  '{"inputs": [{"name": "args", "shape": [1], "datatype": "BYTES", "data": ["audio input data"]}]}'

Understanding tensor remapping

Without the tensorMap, the sentiment model would receive an input tensor named "output" (from whisper), but it expects an input tensor named "args". The mapping whisper.outputs.output: args renames the tensor so the sentiment model receives the correctly named input.

Related Pages

• SeldonIO_Seldon_core_Multi_Modal_Pipeline_Composition -- principle that this implementation realizes
• SeldonIO_Seldon_core_Seldon_Model_CRD_HuggingFace -- depends on deployed HuggingFace model definitions for each pipeline step
• SeldonIO_Seldon_core_Seldon_Model_Infer_BYTES -- uses BYTES datatype for pipeline input
• SeldonIO_Seldon_core_Seldon_Model_CRD_Explainer -- integrates with the explainer model step in the pipeline
• SeldonIO_Seldon_core_Seldon_Pipeline_CRD -- specializes the general Pipeline CRD pattern for multi-modal HuggingFace workflows