Implementation:Kserve Kserve LLMIsvc Manager Deployment

Knowledge Sources	Kserve_Kserve KServe Docs
Domains	Kubernetes, LLM Serving
Last Updated	2026-02-13 00:00 GMT

Overview

Concrete Kubernetes Deployment for the LLMInferenceService controller-manager provided by the KServe project.

Description

This file defines the Kubernetes Deployment for the LLMInferenceService controller-manager, which reconciles LLMInferenceService and LLMInferenceServiceConfig custom resources. The controller runs with leader election, health and readiness probes on port 8081, a webhook server on port 9443, and metrics on port 8443. It is configured with strict security context settings including non-root execution, read-only root filesystem, and all capabilities dropped. The controller manages the lifecycle of LLM-optimized model serving workloads in KServe.

Usage

Apply this Deployment to the kserve namespace to run the LLMInferenceService controller. This is required for managing LLM inference service resources that support disaggregated prefill-decode architectures and optimized LLM serving configurations.

Code Reference

Source Location

Repository: Kserve_Kserve
File: config/llmisvc/manager.yaml
Lines: 1-96

Signature

apiVersion: apps/v1
kind: Deployment
metadata:
  name: llmisvc-controller-manager
  namespace: kserve
  labels:
    app.kubernetes.io/name: llmisvc-controller-manager
    app.kubernetes.io/component: controller
    control-plane: llmisvc-controller-manager
spec:
  selector:
    matchLabels:
      control-plane: llmisvc-controller-manager
  template:
    spec:
      serviceAccountName: $(llmisvc-serviceAccountName)
      containers:
      - name: manager
        image: ko://github.com/kserve/kserve/cmd/llmisvc
        command:
          - /manager
        args:
        - "--metrics-addr=127.0.0.1:8443"
        - "--leader-elect"
        ports:
        - containerPort: 9443
          name: webhook-server
        - containerPort: 8443
          name: metrics

Import

kubectl apply -f config/llmisvc/manager.yaml

I/O Contract

Inputs

Name	Type	Required	Description
$(llmisvc-serviceAccountName)	string (kustomize var)	Yes	ServiceAccount name for the controller pod
llmisvc-webhook-server-cert	Secret	Yes	TLS certificate secret mounted for the webhook server
POD_NAMESPACE	env (fieldRef)	Yes	Injected from the pod metadata namespace

Outputs

Name	Type	Description
llmisvc-controller-manager Deployment	Deployment	Runs the LLMInferenceService controller manager
Webhook server	TCP port 9443	Admission webhook endpoint for LLMInferenceService resources
Metrics endpoint	TCP port 8443	Prometheus metrics endpoint at 127.0.0.1:8443
Health endpoints	HTTP port 8081	Liveness (/healthz) and readiness (/readyz) probe endpoints

Usage Examples

Apply the Deployment

kubectl apply -f config/llmisvc/manager.yaml

Verify the controller is running

kubectl get deployment llmisvc-controller-manager -n kserve
kubectl get pods -n kserve -l control-plane=llmisvc-controller-manager

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment