Environment:Kubeflow Pipelines Python SDK

Knowledge Sources	Kubeflow Pipelines KFP SDK setup.py KFP SDK Docs
Domains	Infrastructure, Python, ML_Pipelines
Last Updated	2026-02-13 13:35 GMT

Overview

Python 3.9+ environment with the kfp SDK package for authoring, compiling, and submitting Kubeflow Pipelines.

Description

This environment provides the Python runtime and kfp SDK required for pipeline development. The SDK includes the DSL for defining components and pipelines, the compiler for generating IR YAML, and the client for submitting runs to a KFP backend. It supports Python 3.9 through 3.13 and depends on Google Cloud libraries, Kubernetes client, protobuf, and YAML processing.

Usage

Use this environment for any Pipeline Authoring, Compilation, or Submission workflow. It is the mandatory prerequisite for running all Python-based implementations including Dsl_Component_Decorator, Dsl_Pipeline_Decorator, Compiler_Compile, Client_Create_Run, and all control flow and XGBoost wrapper implementations.

System Requirements

Category	Requirement	Notes
OS	Linux, macOS, or WSL2	Any OS with Python support
Python	>= 3.9.0	Tested on 3.9, 3.10, 3.11, 3.12, 3.13
Disk	500MB	For SDK and dependencies installation

Dependencies

Python Packages (Core)

kfp (main SDK package)
kfp-pipeline-spec >= 2.15.0, < 3
kfp-server-api >= 2.15.0, < 3
click >= 8.1.8
docstring-parser >= 0.7.3, < 1
google-api-core >= 1.31.5, < 3.0.0dev
google-auth >= 1.6.1, < 3
google-cloud-storage >= 2.2.1, < 4
kubernetes >= 8.0.0, < 31
protobuf >= 6.31.1, < 7.0
PyYAML >= 5.3, < 7
requests-toolbelt >= 0.8.0, < 2
tabulate >= 0.8.6, < 1
urllib3 < 3.0.0
typing-extensions >= 3.7.4, < 5

Optional Extras

kfp[kubernetes]: Installs kfp-kubernetes for PVC, secrets, and node selector support
kfp[notebooks]: Installs nbclient, ipykernel, jupyter_client for notebook execution
kfp[all]: All optional extras combined

Credentials

The following may be required depending on usage:

KFP_PACKAGE_PATH: (Optional) Override the KFP package installed in components; used for testing custom SDK builds.
KFP backend endpoint: The kfp.Client(host=...) requires a reachable KFP API server URL for pipeline submission.

Quick Install

# Install the KFP SDK
pip install kfp

# With Kubernetes platform extensions
pip install "kfp[kubernetes]"

# With all extras
pip install "kfp[all]"

# For development (editable install from source)
pip install -e sdk/python

Code Evidence

Python version requirement from sdk/python/setup.py:104:

python_requires='>=3.9.0',

Supported Python classifiers from sdk/python/setup.py:92-97:

'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',

Client version warning from sdk/python/kfp/client/client.py:159-161:

warnings.warn(
    'This client only works with Kubeflow Pipeline v2.0.0-beta.2 '
    'and later versions.',
    DeprecationWarning,
    stacklevel=2)

Entry points from sdk/python/setup.py:106-110:

entry_points={
    'console_scripts': [
        'dsl-compile = kfp.cli.compile_:main',
        'kfp=kfp.cli.__main__:main',
    ]
}

Common Errors

Error Message	Cause	Solution
`This client only works with Kubeflow Pipeline v2.0.0-beta.2 and later`	KFP backend too old	Upgrade KFP backend to v2.0.0-beta.2+
`The host does not contain the "http" or "https" protocol`	Missing protocol in Client host URL	Prefix host with `http://` or `https://`
`ModuleNotFoundError: No module named 'kfp_pipeline_spec'`	Missing pipeline spec package	`pip install kfp-pipeline-spec>=2.15.0`
`output_component_file parameter is deprecated`	Using old component compilation approach	Use `Compiler().compile()` instead

Compatibility Notes

KFP v1 Incompatibility: The current SDK client only works with KFP v2.0.0-beta.2 and later. V1 pipelines are not supported.
Runtime Mode: When _KFP_RUNTIME=true is set (inside task containers), most SDK imports are disabled. Components must not depend on SDK features at runtime.
kfp-kubernetes: The kfp-kubernetes extension must be installed separately for Kubernetes-specific features (PVCs, secrets, node selectors).
Protobuf: The SDK requires protobuf >= 6.31.1; the kfp-pipeline-spec sub-package pins an exact version (6.31.1).

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment