Environment:Risingwavelabs Risingwave Python Tooling Environment

Knowledge Sources	RisingWave Grafana README
Domains	Infrastructure, Testing, Monitoring
Last Updated	2026-02-09 08:00 GMT

Overview

Python 3.12 environment for Grafana dashboard generation, end-to-end integration test verification scripts, Iceberg compaction utilities, and Python UDF support.

Description

This environment provides the Python runtime and package dependencies used across multiple RisingWave subsystems. Python is used for: (1) generating Grafana dashboard JSON from Python scripts via grafanalib, (2) running end-to-end integration test verification scripts that query sinks, (3) PySpark-based Iceberg table compaction and maintenance scripts, and (4) Python UDF support via PyO3. The CI Docker image pins Python 3.12 as the default and sets PYO3_PYTHON=python3.12.

Usage

Use this environment for generating Grafana dashboards, running integration test verification scripts, Iceberg table maintenance via PySpark, or developing Python UDFs.

System Requirements

Category	Requirement	Notes
OS	Linux, macOS	Windows not tested for Python tooling
Hardware	No special requirements	Standard development machine
Python	3.12 (for PyO3 UDF support)	3.10+ may work for non-UDF tooling

Dependencies

Grafana Dashboard Generation

grafanalib >= 0.7.0
jsonmerge >= 1.9.0

End-to-End Test Scripts

requests
authlib = 1.6.5
confluent_kafka = 2.10.1
protobuf = 5.29.5
fastavro = 1.9.4
httpx = 0.27.2
attrs, cachetools, jsonschema
google-auth, google-api-core, google-cloud-kms
pulsar-client = 3.6.1
psycopg2-binary (for PostgreSQL sink checks)

Python UDF Support

python3.12-dev (system package for development headers)
Environment variable: PYO3_PYTHON=python3.12

Credentials

No specific credentials required for Python tooling. Integration test scripts may connect to external services (Kafka, databases) using connection parameters passed as arguments.

Quick Install

# Install Python 3.12
sudo apt-get install -y python3.12 python3.12-dev python3-pip python3-venv

# Grafana dashboard generation
pip install grafanalib>=0.7.0 jsonmerge>=1.9.0

# End-to-end test dependencies
pip install -r e2e_test/requirements.txt

# For Python UDF support (set in build environment)
export PYO3_PYTHON=python3.12

Code Evidence

Python version pinning from ci/Dockerfile lines 16, 21-22:

RUN apt-get install -y python3.12 python3.12-dev
ENV PYO3_PYTHON=python3.12

Grafana requirements from grafana/requirements.txt lines 1-2:

grafanalib >= 0.7.0
jsonmerge >= 1.9.0

Dashboard generation script usage from grafana/generate.sh lines 7-12:

generate-dashboard -o risingwave-user-dashboard.json risingwave-user-dashboard.dashboard.py
generate-dashboard -o risingwave-dev-dashboard.json risingwave-dev-dashboard.dashboard.py

Common Errors

Error Message	Cause	Solution
`ModuleNotFoundError: No module named 'grafanalib'`	grafanalib not installed	`pip install grafanalib>=0.7.0`
`ImportError: cannot import name 'PyO3'`	Python 3.12 dev headers missing	`sudo apt-get install python3.12-dev`
`psycopg2.OperationalError: connection refused`	Database not running	Start RisingWave or target database before running tests

Compatibility Notes

Python 3.12 Required for PyO3: The Rust-Python bridge (PyO3) is compiled against Python 3.12 specifically. Other Python versions will cause linking errors.
Poetry for Iceberg Tests: Some integration tests (iceberg-sink2, iceberg-source) use Poetry instead of pip for dependency management.
Virtual Environments: Recommended to use python3 -m venv to isolate test dependencies from system Python.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment