Principle:Tensorflow Serving Canary Deployment
| Knowledge Sources | |
|---|---|
| Domains | Deployment, Version_Management |
| Last Updated | 2026-02-13 17:00 GMT |
Overview
A deployment strategy that gradually routes traffic to a new model version alongside the existing stable version, minimizing risk of serving regressions.
Description
Canary deployment in TensorFlow Serving leverages version policies and labels to serve two versions simultaneously. The "stable" label points to the proven version, while "canary" points to the new version. Client-side or infrastructure-level routing directs a small percentage of traffic to the canary. If the canary performs well, the "stable" label is reassigned to the canary version (promotion). If not, the canary is removed (rollback).
The ServerCore orchestrates this by:
- Loading both versions via a Specific version policy
- Assigning labels to each version
- Supporting dynamic configuration reload to update labels without restart
Usage
Use canary deployment when releasing a new model version where incorrect predictions could have significant impact. This pattern requires version labels and specific version policies.
Theoretical Basis
# Abstract canary deployment lifecycle (NOT real implementation)
# Phase 1: Deploy canary alongside stable
config = {
"version_policy": specific([42, 43]),
"labels": {"stable": 42, "canary": 43}
}
server.reload_config(config)
# Phase 2: Monitor canary metrics
if canary_metrics_acceptable():
# Phase 3a: Promote canary to stable
config["labels"] = {"stable": 43}
config["version_policy"] = specific([43])
server.reload_config(config)
else:
# Phase 3b: Rollback - remove canary
config["labels"] = {"stable": 42}
config["version_policy"] = specific([42])
server.reload_config(config)