Implementation:Kubeflow Pipelines Helm Cache Template
| Knowledge Sources | |
|---|---|
| Domains | Deployment, Pipeline_Caching, Helm |
| Last Updated | 2026-02-13 14:00 GMT |
Overview
Helm template for the KFP pipeline caching subsystem resources in the GCP Marketplace deployment, including the cache deployer and cache server.
Description
This template (303 lines) defines the infrastructure for pipeline step caching. It provisions two main components: (1) the cache deployer which sets up TLS certificates and a MutatingWebhookConfiguration to intercept pod creation, and (2) the cache server which handles cache hit/miss logic by comparing task fingerprints against a database. Includes ServiceAccounts, ClusterRoles, Roles, RoleBindings, Deployments, a ConfigMap for database config, and a Service.
Usage
This template is rendered during Helm-based GCP Marketplace installation. The caching subsystem enables pipeline steps to skip re-execution when inputs match a previously completed step.
Code Reference
Source Location
- Repository: Kubeflow_Pipelines
- File: manifests/gcp_marketplace/chart/kubeflow-pipelines/templates/cache.yaml
- Lines: 1-303
Signature
# Key Kubernetes resources defined:
# - ServiceAccount: kubeflow-pipelines-cache-deployer-sa
# - ServiceAccount: kubeflow-pipelines-cache
# - ClusterRole: kubeflow-pipelines-cache-deployer-clusterrole
# - Role: kubeflow-pipelines-cache-deployer-role (secrets management)
# - Role: kubeflow-pipelines-cache-role (pods, configmaps, workflows watch)
# - Deployment: cache-deployer-deployment (TLS cert + webhook setup)
# - Deployment: cache-server (cache hit/miss logic)
# - ConfigMap: cache-configmap (MySQL connection parameters)
# - Service: cache-server (port 443, webhook endpoint)
Import
# Rendered via Helm during GCP Marketplace installation
helm install kubeflow-pipelines chart/kubeflow-pipelines/
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| DBCONFIG_* | ConfigMap keys | Yes | MySQL connection parameters (host, port, user, password, database) |
| CACHE_IMAGE | ConfigMap key | No | Image used for cached step placeholders (default: busybox) |
| CACHE_NODE_RESTRICTIONS | ConfigMap key | No | Boolean flag for node scheduling restrictions |
Outputs
| Name | Type | Description |
|---|---|---|
| MutatingWebhook | side effect | Intercepts pod creation to inject cache checks |
| Cache Server | Service | Endpoint for cache lookup on port 443 |
| TLS certificates | Secret | webhook-server-tls for secure webhook communication |
Usage Examples
Verifying Cache Resources
# Check cache server is running
kubectl get deployment cache-server
# Check cache deployer completed webhook setup
kubectl get mutatingwebhookconfigurations | grep cache
# Verify cache service endpoint
kubectl get service cache-server