Implementation:Kubeflow Pipelines Profile Controller Sync
| Knowledge Sources | |
|---|---|
| Domains | Multi_Tenancy, Kubernetes, Deployment |
| Last Updated | 2026-02-13 14:00 GMT |
Overview
Python HTTP server acting as a Metacontroller sync webhook that generates per-namespace Kubernetes resources for KFP multi-user profile provisioning.
Description
The sync.py module (415 lines) is the central logic for multi-tenant KFP namespace provisioning. It runs as an HTTP server that receives sync requests from Metacontroller containing namespace metadata. For each namespace labeled with `pipelines.kubeflow.org/enabled=true`, it generates: S3/MinIO access secrets (creating per-namespace IAM users via botocore), artifact configuration ConfigMaps, visualization server Deployments and Services, Istio DestinationRules, and AuthorizationPolicies.
Main class: `Controller(BaseHTTPRequestHandler)` — Handles POST requests from Metacontroller, generates namespace-scoped resources.
Environment variables:
- `CONTROLLER_PORT` (default 8080)
- `FRONTEND_IMAGE` / `FRONTEND_TAG`
- `DISABLE_ISTIO_SIDECAR` (boolean)
- `ARTIFACTS_PROXY_ENABLED` (boolean)
- `ARTIFACT_RETENTION_DAYS` (integer, default -1 = disabled)
- `S3_ENDPOINT_URL` (default "http://seaweedfs.kubeflow:8333")
Usage
Deployed as the `pipelines-profile-controller` Deployment in multi-user KFP installations. Metacontroller invokes this webhook whenever a namespace with the pipelines label is created or updated.
Code Reference
Source Location
- Repository: Kubeflow_Pipelines
- File: manifests/kustomize/base/installs/multi-user/pipelines-profile-controller/sync.py
- Lines: 1-415
Signature
class Controller(BaseHTTPRequestHandler):
def sync(self, parent: dict, attachments: dict) -> dict:
"""Generate per-namespace KFP resources based on namespace labels.
Returns dict with 'status' and 'attachments' (child resources)."""
def do_POST(self) -> None:
"""Handle Metacontroller sync webhook POST requests."""
def main() -> None:
"""Entry point: starts HTTP server on CONTROLLER_PORT."""
def get_settings_from_env() -> dict:
"""Load configuration from environment variables."""
def server_factory(port: int) -> HTTPServer:
"""Create HTTPServer instance with Controller handler."""
Import
# Not typically imported; runs as standalone HTTP server
import base64, hashlib, json, os
from http.server import BaseHTTPRequestHandler, HTTPServer
import botocore.session
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| parent | JSON (POST body) | Yes | Metacontroller parent object (namespace metadata) |
| attachments | JSON (POST body) | Yes | Existing child resources in the namespace |
| Environment variables | env vars | Yes | Configuration for images, S3, Istio, retention |
Outputs
| Name | Type | Description |
|---|---|---|
| status | JSON | Sync status for Metacontroller |
| attachments | JSON | Generated Kubernetes resources (Secrets, ConfigMaps, Deployments, Services) |
Usage Examples
Webhook Response Structure
# Example sync response for a pipelines-enabled namespace
{
"status": {"conditions": [{"type": "Ready", "status": "True"}]},
"attachments": [
# Secret: mlpipeline-minio-artifact (S3 credentials)
{"apiVersion": "v1", "kind": "Secret", "metadata": {"name": "mlpipeline-minio-artifact"}},
# ConfigMap: kfp-launcher
{"apiVersion": "v1", "kind": "ConfigMap", "metadata": {"name": "kfp-launcher"}},
# ConfigMap: metadata-grpc-configmap
{"apiVersion": "v1", "kind": "ConfigMap", "metadata": {"name": "metadata-grpc-configmap"}},
]
}