Environment:Kubeflow Kubeflow Python KFP SDK Environment
| Knowledge Sources | |
|---|---|
| Domains | ML_Pipelines, Python, SDK |
| Last Updated | 2026-02-13 00:00 GMT |
Overview
Python 3.9+ environment with Kubeflow Pipelines SDK v2 for defining, compiling, and submitting ML pipelines.
Description
This environment defines the Python runtime and SDK packages required for AI practitioners to interact with Kubeflow Pipelines programmatically. The kfp SDK v2 provides Python decorators for defining pipeline components and DAGs that compile to Argo Workflow or Tekton YAML. Additionally, the kubeflow-katib SDK, kubeflow-training SDK, and kserve SDK provide Python interfaces for hyperparameter tuning, distributed training, and model serving respectively.
Usage
Use this environment for the AI Lifecycle Pipeline workflow, specifically when building pipelines, submitting training jobs, configuring Katib experiments, or deploying InferenceServices from Python code. This is the data scientist's local development environment.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Linux, macOS, or Windows (WSL2) | Linux is the primary development platform |
| Python | >= 3.9 | Required by kfp v2 SDK |
| pip | >= 21.0 | For installing SDK packages |
| Network | Access to Kubeflow API endpoints | Pipeline submission requires cluster connectivity |
Dependencies
Python Packages
- kfp >= 2.0.0 (Kubeflow Pipelines SDK v2)
- kfp-server-api (auto-installed with kfp)
- kubeflow-katib (Katib experiment SDK, optional)
- kubeflow-training (Training Operator SDK, optional)
- kserve (KServe inference SDK, optional)
- model-registry (Model Registry Python client, optional)
Credentials
The following environment variables or configuration may be required:
- KFP_ENDPOINT: URL of the Kubeflow Pipelines API (e.g., https://kubeflow.example.com/pipeline)
- KUBEFLOW_TOKEN: Authentication token for the Kubeflow API (when using multi-user mode with Dex/OIDC)
- KUBECONFIG: For direct cluster access when submitting jobs via kubectl-based SDKs
Quick Install
# Install core Kubeflow SDKs
pip install kfp>=2.0.0
# Install optional component SDKs
pip install kubeflow-katib kubeflow-training kserve model-registry
Code Evidence
KFP SDK v2 pipeline definition pattern (from Build_Pipeline implementation):
from kfp import dsl, compiler
@dsl.component
def train_model(data_path: str, epochs: int) -> str:
# Training logic
return model_path
@dsl.pipeline(name="training-pipeline")
def my_pipeline(data_path: str, epochs: int = 10):
train_task = train_model(data_path=data_path, epochs=epochs)
compiler.Compiler().compile(my_pipeline, "pipeline.yaml")
KFP SDK v2 was introduced in Kubeflow 1.6 (ROADMAP.md:L112-119):
* V2 Preview: Support running pipeline in Argo-agnostic approach
* SDK to change from `kfp.v2` to `kfp`
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
ModuleNotFoundError: No module named 'kfp' |
kfp SDK not installed | pip install kfp>=2.0.0
|
kfp.Client: connection refused |
Kubeflow Pipelines API unreachable | Verify KFP_ENDPOINT and network connectivity; check port-forward if needed |
Unauthorized: token expired |
OIDC token expired in multi-user mode | Refresh authentication token via Dex login flow |
ImportError: cannot import name 'dsl' from 'kfp' |
Using kfp v1 syntax with v2 install | Migrate to kfp v2 API: from kfp import dsl (not from kfp.v2 import dsl)
|
Compatibility Notes
- kfp v1 vs v2: Kubeflow Pipelines 2.0+ uses the kfp v2 SDK. The v1 SDK is deprecated. Pipeline YAML compiled with v1 is not compatible with v2 backend.
- Tekton backend: Kubeflow 1.2+ supports an optional Tekton backend. KFP v2 merged kfp-tekton support into a single repository (ROADMAP.md v1.9).
- Notebook integration: The KFP SDK can be used directly from Kubeflow Notebooks (Jupyter, VS Code) running inside the cluster, which simplifies authentication.