Principle:Kserve Kserve Canary Validation

Knowledge Sources	Canary Testing KServe Tag Routing
Domains	MLOps, Testing, Traffic_Management
Last Updated	2026-02-13 00:00 GMT

Overview

A testing pattern that validates canary model versions using tag-based routing to send deterministic test traffic to specific revisions.

Description

Canary Validation uses KServe's tag-based routing feature to send test requests directly to either the canary or stable revision, bypassing the probabilistic traffic split. This enables deterministic testing of the new model version before increasing its traffic share.

When the annotation serving.kserve.io/enable-tag-routing: "true" is set, Knative creates tagged routes:

• http://latest-<name>.../ routes exclusively to the canary revision
• http://prev-<name>.../ routes exclusively to the stable revision

Usage

Use this technique after deploying a canary split to validate the new model version with known test inputs. Compare canary predictions against baseline predictions to assess accuracy, latency, and error rates before increasing traffic.

Theoretical Basis

# Tag-based routing model (NOT implementation code)
Annotation: serving.kserve.io/enable-tag-routing: "true"

Tagged endpoints:
  latest-<name>.<namespace>.example.com  → canary revision only
  prev-<name>.<namespace>.example.com    → stable revision only
  <name>.<namespace>.example.com         → probabilistic split (P% / (100-P)%)

Validation workflow:
1. Send test data to latest-<name> endpoint
2. Compare predictions against ground truth or stable version
3. Check latency, error rate, confidence scores
4. If acceptable → increase canaryTrafficPercent
5. If degraded → rollback (set canaryTrafficPercent: 0)

Related Pages

Implemented By

• Implementation:Kserve_Kserve_Tag_Based_Routing_Validation