Implementation:Kserve Kserve MMS Predictor Detection

Knowledge Sources	KServe KServe Multi-Model Serving
Domains	MLOps, Cost_Optimization, Model_Serving
Last Updated	2026-02-13 00:00 GMT

Overview

Concrete Go utility functions for detecting multi-model serving mode and creating empty model configuration ConfigMaps.

Description

IsMMSPredictor() checks whether an InferenceService predictor is configured for multi-model serving by verifying the absence of storage configurations. IsMemoryResourceAvailable() validates that the shared pod has enough memory to host the requested TrainedModel. CreateEmptyModelConfig() initializes the ConfigMap that the model agent watches.

Usage

These functions are called automatically during reconciliation. The MMS detection drives whether the controller creates empty model ConfigMaps and enables the agent sidecar.

Code Reference

Source Location

• Repository: kserve
• File: pkg/controller/v1beta1/inferenceservice/utils/utils.go, Lines 55-91
• File: pkg/modelconfig/configmap.go, Lines 119-133 (CreateEmptyModelConfig)

Signature

// IsMMSPredictor checks if the predictor is in multi-model serving mode
func IsMMSPredictor(predictor *v1beta1.PredictorSpec) bool

// IsMemoryResourceAvailable checks if the pod has enough memory for the model
func IsMemoryResourceAvailable(isvc *v1beta1.InferenceService,
    totalReqMemory resource.Quantity) bool

// CreateEmptyModelConfig creates the initial empty ConfigMap for MMS
func CreateEmptyModelConfig(clientset kubernetes.Interface,
    namespace string, name string) error

Import

import (
    "github.com/kserve/kserve/pkg/controller/v1beta1/inferenceservice/utils"
    "github.com/kserve/kserve/pkg/modelconfig"
)

I/O Contract

Inputs

Name	Type	Required	Description
predictor	*v1beta1.PredictorSpec	Yes	Predictor spec to check for MMS mode
totalReqMemory	resource.Quantity	Yes (memory check)	Sum of all TrainedModel memory requests

Outputs

Name	Type	Description
IsMMSPredictor	bool	true if no storageUri/storageSpec and not HuggingFace
IsMemoryResourceAvailable	bool	true if pod memory limit >= total requested memory
ConfigMap	v1.ConfigMap	Empty modelconfig-<name>-0 ConfigMap

Usage Examples

MMS InferenceService (No storageUri)

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: sklearn-iris-example
spec:
  predictor:
    minReplicas: 1
    sklearn:
      protocolVersion: v1
      resources:
        limits:
          memory: 512Mi
# No storageUri → IsMMSPredictor() returns true
# Controller creates ConfigMap: modelconfig-sklearn-iris-example-0

# Verify MMS ConfigMap created
kubectl get configmap modelconfig-sklearn-iris-example-0
# NAME                                DATA   AGE
# modelconfig-sklearn-iris-example-0  0      30s

Related Pages

Implements Principle

• Principle:Kserve_Kserve_Multi_Model_Shared_Service