Principle:Kserve Kserve LLMIsvc Installation
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Kubernetes, LLM_Serving |
| Last Updated | 2026-02-13 00:00 GMT |
Overview
An installation procedure that deploys the LLMInferenceService subsystem including controller, CRDs, networking stack, and scheduler components.
Description
The LLMIsvc Installation is a multi-component deployment that sets up the infrastructure for LLM inference serving. Unlike the base KServe installation, the LLMIsvc subsystem requires additional components:
- cert-manager: TLS certificate management for webhooks
- Gateway API CRDs: Kubernetes networking primitives
- Envoy Gateway + AI Gateway: Ingress and AI-aware routing
- Leader Worker Set (LWS): Multi-pod coordination for distributed inference
- KServe LLMIsvc Helm chart: Controller, CRDs, ConfigMaps
Usage
Run this installation once per cluster before deploying any LLMInferenceService resources. The quick install script handles the full dependency chain.
Theoretical Basis
# Installation dependency chain (NOT implementation code)
1. cert-manager → TLS for webhooks
2. Gateway API CRDs → HTTPRoute, GatewayClass
3. Envoy Gateway → Ingress controller
4. Envoy AI Gateway → AI-aware routing
5. Leader Worker Set → Multi-node model serving
6. MetalLB (minikube) → LoadBalancer IP allocation
7. GatewayClass → "envoy" gateway class
8. KServe LLMIsvc chart → Controller, CRDs, webhooks
9. Gateway resource → kserve-ingress-gateway on port 80
Related Pages
Implemented By
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment