Heuristic:Kubeflow Kubeflow Kustomize Build Pipe Apply Pattern
| Knowledge Sources | |
|---|---|
| Domains | Platform_Deployment, Kubernetes, Configuration_Management |
| Last Updated | 2026-02-13 00:00 GMT |
Overview
Use kustomize build | kubectl apply -f - instead of kubectl apply -k for reliable Kubeflow manifest deployment.
Description
The Kubeflow project standardized on the standalone kustomize binary piped to kubectl apply as the canonical deployment pattern. This two-step approach separates manifest rendering from application, providing better error diagnostics and compatibility guarantees. The alternative kubectl apply -k uses an embedded kustomize version that often lags behind the standalone release and may not support features used in Kubeflow overlays.
Usage
Use this pattern for every Kubeflow component deployment, including cert-manager, Istio, Dex, and all application components. This heuristic was established when kustomize replaced ksonnet in Kubeflow 0.6 (ROADMAP.md) and has been the standard ever since.
The Insight (Rule of Thumb)
- Action: Always use the two-step pattern:
kustomize build {path} | kubectl apply -f - - Value: Standalone kustomize ensures feature compatibility; piping allows inspection of rendered YAML before apply.
- Trade-off: Requires installing standalone kustomize in addition to kubectl. Minor inconvenience for significantly better reliability.
- Debug tip: Run
kustomize build {path}alone to inspect the rendered YAML before applying. This catches kustomization errors without affecting the cluster.
Reasoning
Kubeflow manifests use advanced kustomize features such as:
- Strategic merge patches for configuring Istio, Dex, and component resources
- Overlay hierarchies (base → overlay → distribution-specific overlay)
- Replacement transformers for injecting configuration across resources
The kubectl embedded kustomize may not support all these features. The Kubeflow project transitioned from ksonnet to kustomize in version 0.6 (ROADMAP.md:L413):
* kustomize replaced ksonnet for application configuration
The standard deployment commands throughout the kubeflow/manifests repository consistently use:
kustomize build common/cert-manager/cert-manager/base | kubectl apply -f -
kustomize build common/istio-1-22/istio-crds/base | kubectl apply -f -
kustomize build apps/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user | kubectl apply -f -
Never:
# DO NOT use this - embedded kustomize may be outdated
kubectl apply -k common/cert-manager/cert-manager/base