Implementation:Kserve Kserve Model Agent Pipeline

Knowledge Sources	KServe KServe Multi-Model Serving
Domains	MLOps, Model_Management, Storage
Last Updated	2026-02-13 00:00 GMT

Overview

Concrete Go implementation of the Watcher, Puller, and Downloader components that form the model agent sidecar pipeline.

Description

The model agent consists of three cooperating components in pkg/agent/:

• Watcher (watcher.go): Uses fsnotify to detect ConfigMap mount changes, parses model config JSON, computes diffs using the stale flag pattern.
• Puller (puller.go): Runs per-model goroutines via modelProcessor() to handle add/remove operations asynchronously.
• Downloader (downloader.go): Downloads from S3/GCS/HTTP using provider-specific clients, with SHA256-based idempotency.

Usage

The agent runs as a sidecar container in MMS pods. It is automatically injected by the pod mutating webhook.

Code Reference

Source Location

• Repository: kserve
• File: pkg/agent/watcher.go, Lines 34-197
• File: pkg/agent/puller.go, Lines 41-223
• File: pkg/agent/downloader.go, Lines 34-97
• File: pkg/agent/syncer.go, Lines 36-77

Signature

// Watcher monitors ConfigMap for model config changes
type Watcher struct {
    configDir    string
    modelTracker map[string]ModelConfig
    puller       *Puller
}
func (w *Watcher) Start() error

// Puller processes model download/load and remove/unload operations
type Puller struct {
    channelMap   map[string]ModelOp
    completions  chan ModelOp
    opStats      map[string]ModelOpStats
    Downloader   *Downloader
    logger       *zap.SugaredLogger
}
func (p *Puller) StartPullerAndProcessModels(channels *ModelChannels)

// Downloader downloads model artifacts from cloud storage
type Downloader struct {
    ModelDir  string
    Providers map[storage.Protocol]storage.Provider
    Logger    *zap.SugaredLogger
}
func (d *Downloader) DownloadModel(modelName string,
    storageUri string) error

Import

import "github.com/kserve/kserve/pkg/agent"

I/O Contract

Inputs

Name	Type	Required	Description
configDir	string	Yes	ConfigMap mount path (watched by fsnotify)
ModelConfig JSON	JSON	Yes	Model specifications from ConfigMap

Outputs

Name	Type	Description
Downloaded models	Files	Model artifacts in local modelDir
V2 load calls	HTTP POST	/v2/repository/models/{name}/load to model server
V2 unload calls	HTTP POST	/v2/repository/models/{name}/unload to model server
SUCCESS markers	Files	SUCCESS.<sha256> files for idempotency

Usage Examples

Agent Lifecycle

# Agent runs automatically in MMS pods
# Monitor agent logs:
kubectl logs <pod-name> -c agent

# Expected log sequence for model load:
# "Detected config change"
# "Downloading model model1-sklearn from gs://..."
# "Successfully downloaded model"
# "Loading model model1-sklearn"
# "Model loaded successfully"

Related Pages

Implements Principle

• Principle:Kserve_Kserve_Model_Agent_Loading