Implementation:Kserve Kserve LocalModelNode Agent DaemonSet

Knowledge Sources	Kserve_Kserve KServe Docs
Domains	Kubernetes, Model Caching
Last Updated	2026-02-13 00:00 GMT

Overview

Concrete Kubernetes DaemonSet for the LocalModelNode agent provided by the KServe project.

Description

This file defines the Kubernetes DaemonSet for the LocalModelNode agent, which runs on labeled worker nodes to manage local model file caching. The DaemonSet uses a node selector (kserve/localmodel: worker) to target specific nodes and runs a manager container with a host-path volume mount at /mnt/models mapped to /models on the host. It runs as non-root with strict security context settings, with resource limits of 100m CPU and 300Mi memory. The agent downloads and manages model artifacts directly on the node filesystem for faster model loading.

Usage

Apply this DaemonSet to the kserve namespace after labeling the target nodes with kserve/localmodel=worker. The agent will automatically deploy to labeled nodes and begin managing local model storage. This is required for the local model caching feature that reduces inference cold-start time by pre-caching model files on worker nodes.

Code Reference

Source Location

Repository: Kserve_Kserve
File: config/localmodelnodes/manager.yaml
Lines: 1-70

Signature

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kserve-localmodelnode-agent
  namespace: kserve
  labels:
    app.kubernetes.io/name: kserve-localmodelnode-agent
    control-plane: kserve-localmodelnode-agent
spec:
  selector:
    matchLabels:
      control-plane: kserve-localmodelnode-agent
  template:
    spec:
      nodeSelector:
        kserve/localmodel: worker
      serviceAccountName: kserve-localmodelnode-agent
      containers:
      - command:
        - /manager
        image: ko://github.com/kserve/kserve/cmd/localmodelnode
        name: manager
        env:
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
        volumeMounts:
          - mountPath: /mnt/models
            name: models
      volumes:
        - name: models
          hostPath:
            path: /models
            type: DirectoryOrCreate

Import

kubectl apply -f config/localmodelnodes/manager.yaml

I/O Contract

Inputs

Name	Type	Required	Description
Node label kserve/localmodel=worker	Node label	Yes	Determines which nodes run the agent via nodeSelector
POD_NAMESPACE	env (fieldRef)	Yes	Injected from the pod metadata namespace
NODE_NAME	env (fieldRef)	Yes	Injected from the pod spec nodeName to identify the current node
kserve-localmodelnode-agent	ServiceAccount	Yes	ServiceAccount used by the agent pods

Outputs

Name	Type	Description
kserve-localmodelnode-agent DaemonSet	DaemonSet	Runs the local model node agent on labeled worker nodes
/models host directory	Host filesystem	Model artifacts cached on the node at /models (DirectoryOrCreate)

Usage Examples

Label nodes for local model caching

kubectl label node my-gpu-node kserve/localmodel=worker

Apply the DaemonSet

kubectl apply -f config/localmodelnodes/manager.yaml

Verify agent pods are running

kubectl get daemonset kserve-localmodelnode-agent -n kserve
kubectl get pods -n kserve -l control-plane=kserve-localmodelnode-agent

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment