Implementation:Kserve Kserve LocalModel Manager Deployment

Knowledge Sources	Kserve_Kserve KServe Docs
Domains	Kubernetes, Model Caching
Last Updated	2026-02-13 00:00 GMT

Overview

Concrete Kubernetes Deployment for the LocalModel controller-manager provided by the KServe project.

Description

This file defines the Kubernetes Deployment for the LocalModel controller-manager, which reconciles LocalModelCache custom resources to manage model caching across cluster nodes. The controller deploys as a single replica with health and readiness probes on port 8081, a webhook server on port 9443, and TLS certificate volume mounts from the localmodel-webhook-server-cert secret. It runs with strict security context settings (non-root, read-only filesystem, dropped capabilities), limited to 100m CPU and 300Mi memory. This controller orchestrates model caching by managing LocalModelCache and LocalModelNode resources.

Usage

Apply this Deployment to the kserve namespace to run the LocalModel controller. This is required for the local model caching subsystem, which coordinates model pre-caching across designated node groups to reduce inference cold-start latency.

Code Reference

Source Location

Repository: Kserve_Kserve
File: config/localmodels/manager.yaml
Lines: 1-83

Signature

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kserve-localmodel-controller-manager
  namespace: kserve
  labels:
    app.kubernetes.io/name: kserve-localmodel-controller-manager
    control-plane: kserve-localmodel-controller-manager
spec:
  selector:
    matchLabels:
      control-plane: kserve-localmodel-controller-manager
  template:
    spec:
      serviceAccountName: kserve-localmodel-controller-manager
      containers:
      - command:
        - /manager
        image: ko://github.com/kserve/kserve/cmd/localmodel
        name: manager
        ports:
        - containerPort: 9443
          name: webhook-server
        volumeMounts:
        - mountPath: /tmp/k8s-webhook-server/serving-certs
          name: cert
      volumes:
      - name: cert
        secret:
          defaultMode: 420
          secretName: localmodel-webhook-server-cert

Import

kubectl apply -f config/localmodels/manager.yaml

I/O Contract

Inputs

Name	Type	Required	Description
kserve-localmodel-controller-manager	ServiceAccount	Yes	ServiceAccount for the controller pod
localmodel-webhook-server-cert	Secret	Yes	TLS certificate secret for the webhook server
POD_NAMESPACE	env (fieldRef)	Yes	Injected from the pod metadata namespace

Outputs

Name	Type	Description
kserve-localmodel-controller-manager Deployment	Deployment	Runs the LocalModel controller that manages model caching
Webhook server	TCP port 9443	Admission webhook endpoint for LocalModel resources
Health endpoints	HTTP port 8081	Liveness (/healthz) and readiness (/readyz) probe endpoints

Usage Examples

Apply the Deployment

kubectl apply -f config/localmodels/manager.yaml

Verify the controller is running

kubectl get deployment kserve-localmodel-controller-manager -n kserve
kubectl get pods -n kserve -l control-plane=kserve-localmodel-controller-manager

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment