Implementation:Kserve Kserve InferenceRouterType Enum

Knowledge Sources	KServe KServe InferenceGraph
Domains	Pipeline, Architecture, Graph_Theory
Last Updated	2026-02-13 00:00 GMT

Overview

Concrete Go enum definition for the four InferenceGraph routing types: Sequence, Splitter, Ensemble, and Switch.

Description

The InferenceRouterType enum defines the available routing patterns for InferenceGraph nodes. Each type determines how the graph router dispatches requests to the node's steps. The enum is defined in the InferenceGraph CRD types and validated during graph admission.

Usage

Set the routerType field on each node in the InferenceGraph spec. The root node must always be defined, and its router type determines the top-level dispatch pattern.

Code Reference

Source Location

• Repository: kserve
• File: pkg/apis/serving/v1alpha1/inference_graph.go, Lines 98-112

Signature

// InferenceRouterType defines the routing behavior of a graph node
type InferenceRouterType string

const (
    // Sequence chains steps serially
    Sequence InferenceRouterType = "Sequence"
    // Splitter routes to one step based on weighted random
    Splitter InferenceRouterType = "Splitter"
    // Ensemble fans out to all steps and merges responses
    Ensemble InferenceRouterType = "Ensemble"
    // Switch routes based on GJSON conditions
    Switch   InferenceRouterType = "Switch"
)

Import

import servingv1alpha1 "github.com/kserve/kserve/pkg/apis/serving/v1alpha1"

I/O Contract

Inputs

Name	Type	Required	Description
routerType	InferenceRouterType	Yes	One of: Sequence, Splitter, Ensemble, Switch

Outputs

Name	Type	Description
Routing behavior	function	Determines how the router dispatches to steps

Usage Examples

Ensemble Node

apiVersion: serving.kserve.io/v1alpha1
kind: InferenceGraph
metadata:
  name: model-ensemble
spec:
  nodes:
    root:
      routerType: Ensemble
      steps:
        - serviceName: sklearn-iris
          name: sklearn-iris
        - serviceName: xgboost-iris
          name: xgboost-iris

Switch Node with GJSON Conditions

nodes:
  root:
    routerType: Switch
    steps:
      - serviceName: model-a
        name: model-a
        condition: "instances.#(target==\"a\")"
      - serviceName: model-b
        name: model-b
        condition: "instances.#(target==\"b\")"

Related Pages

Implements Principle

• Principle:Kserve_Kserve_Graph_Topology_Design