Environment:Kserve Kserve Gateway API
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Networking |
| Last Updated | 2026-02-13 14:00 GMT |
Overview
Kubernetes Gateway API v1.3.0 with Envoy Gateway v1.5.0 providing ingress routing for LLMInferenceService deployments.
Description
The Gateway API is an alternative to Istio VirtualService for routing external traffic to KServe services. It is the primary networking backend for the LLMInferenceService (LLMIsvc) controller, using HTTPRoute resources and the Gateway Inference Extension (GIE) v1.2.0 for inference-aware routing. Envoy Gateway serves as the data plane implementation.
Usage
Use this environment for LLMInferenceService deployments or as an alternative to Istio for standard InferenceService ingress routing.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| Kubernetes | >= 1.24 | Base requirement |
| Gateway API CRDs | v1.3.0 | Standard Gateway API resources |
| GIE CRDs | v1.2.0 | Gateway Inference Extension |
| Envoy Gateway | v1.5.0 | Gateway API implementation |
| Envoy AI Gateway | v0.4.0 | Optional AI-specific extensions |
Dependencies
CRD Installations
- Gateway API standard CRDs
- Gateway Inference Extension CRDs (InferencePool, InferenceModel)
Credentials
No additional credentials required.
Quick Install
# Install Gateway API CRDs
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/${GATEWAY_API_VERSION}/standard-install.yaml
# Install Gateway Inference Extension CRDs
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases/download/${GIE_VERSION}/manifests.yaml
# Install Envoy Gateway
helm install eg oci://docker.io/envoyproxy/gateway-helm \
--version ${ENVOY_GATEWAY_VERSION} -n envoy-gateway-system --create-namespace
Code Evidence
Versions from `kserve-deps.env:18-19,38-40`:
ENVOY_GATEWAY_VERSION=v1.5.0
ENVOY_AI_GATEWAY_VERSION=v0.4.0
GATEWAY_API_VERSION=v1.3.0
GIE_VERSION=v1.2.0
Go module dependency from `go.mod`:
sigs.k8s.io/gateway-api v1.3.1-0.20251106052652-079e4774d76b
sigs.k8s.io/gateway-api-inference-extension v1.2.0
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `no matching GatewayClass` | Envoy Gateway not installed | Install Envoy Gateway Helm chart |
| HTTPRoute not reconciled | GIE CRDs missing | Apply Gateway Inference Extension manifests |
Compatibility Notes
- Istio alternative: Can replace Istio for ingress when using RawDeployment or LLMIsvc
- LLMIsvc requirement: Gateway API is the default networking for LLMInferenceService
- InferencePool: GIE provides InferencePool and InferenceModel CRDs for inference-aware load balancing
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment