Principle:Kubeflow Pipelines Component Definition

Metadata

Knowledge Sources	Kubeflow Pipelines SDK KFP Component Authoring
Domains	ML_Pipelines Component_Authoring
Last Updated	2026-02-13 00:00 GMT

Overview

A design pattern that encapsulates a unit of ML pipeline logic into a self-contained, reusable building block with typed inputs and outputs. In Kubeflow Pipelines, component definition is the foundational abstraction that enables composable, reproducible machine learning workflows.

Description

Component definition in KFP revolves around how the @dsl.component and @dsl.container_component decorators transform ordinary Python functions into pipeline components. Components are the fundamental building blocks of any Kubeflow pipeline -- each component defines typed inputs and outputs and encapsulates a discrete unit of logic.

There are two primary flavors of component:

• Python (lightweight) components -- defined by decorating a self-contained Python function with @dsl.component. The function body must be serializable and may only reference packages explicitly listed via packages_to_install or available in the base image.
• Container components -- defined by decorating a function with @dsl.container_component. The function returns a dsl.ContainerSpec specifying a custom container image, command, and arguments. This is suitable for logic that depends on complex or non-Python dependencies.

The type system underpinning component I/O includes:

• Primitive types: str, int, float, bool
• Collection types: Dict, List
• Artifact types: Dataset, Model, Metrics, and others, accessed through Input[T], Output[T], InputPath, and OutputPath handles

This type system enables the pipeline compiler to validate connections between components and to manage data passing (small parameters inline, large artifacts via object storage).

Usage

Define a component when you need a reusable pipeline step with an explicit I/O contract. The choice between the two component styles depends on the complexity of your logic:

• Use lightweight Python components (@dsl.component) when the step logic can be expressed as a self-contained Python function with a small set of pip-installable dependencies.
• Use container components (@dsl.container_component) when the step requires a custom container image, non-Python tooling, or complex system-level dependencies.

In both cases, the decorator serializes the function into a container-executable component specification that the KFP compiler can embed into a pipeline DAG.

Theoretical Basis

The component definition pattern is an application of the component abstraction paradigm: encapsulate logic behind a well-defined, typed interface so that consumers need not know the implementation details.

The abstract pattern is as follows:

COMPONENT(name, interface, implementation):
    INTERFACE:
        INPUTS:
            param_1 : Type_A        -- consumed as function parameter
            param_2 : Artifact_B    -- consumed via Input[B] or InputPath('B')
        OUTPUTS:
            result  : Type_C        -- produced via return value
            artifact: Artifact_D    -- produced via Output[D] or OutputPath('D')

    IMPLEMENTATION:
        Given inputs, execute encapsulated logic, produce outputs.

    SERIALIZATION:
        Decorator captures function source + metadata ->
            ComponentSpec (YAML / IR) ->
                Container image entry point at runtime.

Key theoretical properties:

• Encapsulation -- internal logic is hidden; only the typed interface is visible to the pipeline graph.
• Composability -- components with compatible I/O types can be wired together to form arbitrarily complex DAGs.
• Reproducibility -- each component runs in its own container with a pinned image and explicit dependencies, ensuring deterministic execution.

Related Pages

• Implementation:Kubeflow_Pipelines_Dsl_Component_Decorator