Implementation:Apache Paimon Lint Python Script

Knowledge Sources	Apache_Paimon
Domains	CI/CD, Code Quality
Last Updated	2026-02-08 00:00 GMT

Overview

lint-python.sh is the primary CI/CD entry point for the Paimon Python SDK, orchestrating code quality checks (flake8), unit tests (pytest), torch integration tests, and Java-Python interoperability tests.

Description

This script provides a modular check system where each check is implemented as a function suffixed with `_check` (e.g., `flake8_check`, `pytest_check`, `pytest_torch_check`, `mixed_check`). The script discovers all check functions at runtime via `declare -F` and builds a registry in the `SUPPORT_CHECKS` array. Users can selectively run checks via `-i` (include) or `-e` (exclude) flags. The flake8 check scans all Python files (excluding `.tox` and `.venv`) and applies style rules from `dev/cfg.ini`. The pytest check auto-detects the Python version and routes to `py36/` tests for Python 3.6 or standard tests for other versions, excluding e2e and torch tests. The pytest_torch check runs torch-specific integration tests. The mixed check delegates to `run_mixed_tests.sh` for comprehensive Java-Python interoperability testing. All output is logged to `dev/log/paimon-{user}-python-{hostname}.log` with colored terminal output for immediate feedback.

This modular architecture makes it easy to add new checks and provides fine-grained control over which checks to run during development versus CI.

Usage

This script is invoked during CI/CD pipelines and can be run locally by developers to validate changes before committing.

Code Reference

Source Location

• Repository: Apache_Paimon
• File: paimon-python/dev/lint-python.sh

Signature

#!/usr/bin/env bash

# Key functions
function print_function() { ... }
function flake8_check() { ... }
function pytest_check() { ... }
function pytest_torch_check() { ... }
function mixed_check() { ... }
function get_all_supported_checks() { ... }
function collect_checks() { ... }
function check_stage() { ... }

Import

# Run from paimon-python directory
cd paimon-python
./dev/lint-python.sh

I/O Contract

Inputs

Name	Type	Required	Description
-e	String (comma-separated)	no	Checks to exclude (e.g., "flake8,pytest")
-i	String (comma-separated)	no	Checks to include (e.g., "flake8")
-l	Flag	no	List all supported checks

Outputs

Name	Type	Description
Log file	File	Detailed results in `dev/log/paimon-{user}-python-{hostname}.log`
Exit code	Integer	0 on success, 1 on failure
Terminal output	stdout/stderr	Colored status messages

Usage Examples

Run All Checks

# Run all checks (default)
./dev/lint-python.sh

# Expected output:
# =================== checks starting ===================
# =================== flake8 checks ===================
# =================== flake8 checks... [SUCCESS] ===================
# ...

Run Specific Checks

# Run only flake8 and pytest
./dev/lint-python.sh -i flake8,pytest

# Exclude mixed tests (faster for local dev)
./dev/lint-python.sh -e mixed

# List available checks
./dev/lint-python.sh -l

CI Integration

# GitHub Actions workflow
- name: Run Python linting and tests
  run: |
    cd paimon-python
    ./dev/lint-python.sh

Related Pages

• Principle:Apache_Paimon_Developer_Tooling