Implementation:Apache Paimon DlfProvider

Knowledge Sources	Apache_Paimon
Domains	Authentication, Cloud Integration
Last Updated	2026-02-08 00:00 GMT

Overview

DlfProvider implements Data Lake Formation (DLF) authentication for Apache Paimon's REST API, supporting Alibaba Cloud DLF service integration with automatic token refresh.

Description

The DlfProvider module provides DLFAuthProvider, an AuthProvider implementation that handles authentication for Alibaba Cloud's Data Lake Formation service. It manages DLF access tokens, automatically refreshes expired tokens, and signs HTTP requests using either the default DLF signing algorithm or the OpenAPI signing algorithm based on the service endpoint.

The provider maintains a token lifecycle by checking token expiration times and automatically reloading tokens through a DLFTokenLoader when they approach expiration (within 1 hour). It extracts the host from the REST endpoint URI and uses it along with regional information and security tokens to generate proper authentication signatures.

DLFAuthProvider supports two signing algorithms: the standard DLF default signer for most DLF endpoints, and the OpenAPI signer for next-generation DLF endpoints (dlfnext). The provider constructs signed headers including timestamp, security token information, and authorization signatures that comply with DLF's authentication requirements. Token management can be either static (provided at initialization) or dynamic (loaded via DLFTokenLoader), making it suitable for both short-lived applications and long-running services.

Usage

Use DlfProvider when connecting Apache Paimon to Alibaba Cloud DLF-based catalogs, implementing DLF authentication for REST catalog clients, or managing DLF tokens with automatic refresh for long-running applications.

Code Reference

Source Location

• Repository: Apache_Paimon
• File: paimon-python/pypaimon/api/auth/dlf_provider.py

Signature

class DLFAuthProvider(AuthProvider):
    DLF_AUTHORIZATION_HEADER_KEY = "Authorization"
    TOKEN_EXPIRATION_SAFE_TIME_MILLIS = 3_600_000

    def __init__(self,
                 uri: str,
                 region: str,
                 signing_algorithm: str,
                 token: DLFToken = None,
                 token_loader: DLFTokenLoader = None):
        pass

    def _create_signer(self, signing_algorithm: str) -> DLFRequestSigner:
        pass

    @staticmethod
    def extract_host(uri: str) -> str:
        pass

    def get_token(self) -> DLFToken:
        pass

    def merge_auth_header(
            self, base_header: Dict[str, str], rest_auth_parameter: RESTAuthParameter
    ) -> Dict[str, str]:
        pass

Import

from pypaimon.api.auth.dlf_provider import DLFAuthProvider

I/O Contract

Inputs

Name	Type	Required	Description
uri	str	Yes	DLF service endpoint URI
region	str	Yes	Alibaba Cloud region (e.g., "cn-hangzhou")
signing_algorithm	str	Yes	Signing algorithm: "default" or "openapi"
token	DLFToken	No	Static DLF token (either token or token_loader required)
token_loader	DLFTokenLoader	No	Dynamic token loader (either token or token_loader required)
base_header	Dict[str, str]	Yes	Base HTTP headers to augment
rest_auth_parameter	RESTAuthParameter	Yes	REST request parameters for signing

Outputs

Name	Type	Description
headers	Dict[str, str]	HTTP headers with DLF authentication signature
token	DLFToken	Current valid DLF token

Usage Examples

from pypaimon.api.auth.dlf_provider import DLFAuthProvider
from pypaimon.api.token_loader import DLFToken, DLFECSTokenLoader
from pypaimon.api.typedef import RESTAuthParameter

# Create with static token
token = DLFToken(
    access_key_id="your-access-key-id",
    access_key_secret="your-access-key-secret",
    security_token="your-security-token",
    expiration="2026-02-09T00:00:00Z"
)

provider = DLFAuthProvider(
    uri="https://dlf.cn-hangzhou.aliyuncs.com",
    region="cn-hangzhou",
    signing_algorithm="default",
    token=token
)

# Or create with token loader for automatic refresh
token_loader = DLFECSTokenLoader(
    ecs_metadata_url="http://100.100.100.200/latest/meta-data/Ram/security-credentials/",
    role_name="my-role"
)

provider = DLFAuthProvider(
    uri="https://dlf.cn-hangzhou.aliyuncs.com",
    region="cn-hangzhou",
    signing_algorithm="default",
    token_loader=token_loader
)

# Use provider to authenticate requests
base_headers = {"Content-Type": "application/json"}
auth_param = RESTAuthParameter(data={"method": "GET", "path": "/databases"})

authenticated_headers = provider.merge_auth_header(base_headers, auth_param)
# Headers now include Authorization, timestamp, and security token fields

# Extract host from URI
host = DLFAuthProvider.extract_host("https://dlf.cn-hangzhou.aliyuncs.com/v1/catalogs")
# Returns: "dlf.cn-hangzhou.aliyuncs.com"

Related Pages

• Principle:Apache_Paimon_Authentication