Environment:ClickHouse ClickHouse Python3 Test Environment
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Testing |
| Last Updated | 2026-02-08 18:00 GMT |
Overview
Python 3 environment with optional `jinja2` and `termcolor` packages for running the ClickHouse stateless test suite via the `clickhouse-test` runner.
Description
The ClickHouse test runner (`tests/clickhouse-test`) is a 5,270-line Python 3 script that orchestrates stateless functional tests. It manages server startup, test execution, reference comparison, blacklist filtering, and parallel test scheduling. The runner creates temporary databases with random names and normalizes output before comparing with `.reference` files. Optional Python packages enhance functionality: `jinja2` enables template-based test generation and `termcolor` provides colored terminal output.
Usage
Use this environment for any Running Stateless Tests workflow or when writing and validating new tests. It is the mandatory prerequisite for the `clickhouse-test` runner, integration test execution via Praktika, and local test development.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Linux, macOS | Tests primarily target Linux; some tests have platform-specific behavior |
| Software | Python 3 | Shebang: `#!/usr/bin/env python3`; no specific minimum version beyond 3.x |
| Software | Running ClickHouse server | Tests connect to a local server instance |
| Disk | 1GB+ | Test data, temporary databases, and log files |
Dependencies
System Packages
- `python3`
- `clickhouse` binary (server + client, built from source or installed)
Python Packages
- `jinja2` (optional, for template-based test generation)
- `termcolor` (optional, for colored test output)
- Standard library modules used: `json`, `multiprocessing`, `subprocess`, `argparse`, `datetime`, `ssl`, `http.client`
Credentials
No credentials required for local test execution. CI test environments may need:
- `CLICKHOUSE_HOST`: Server hostname (default: `localhost`)
- `CLICKHOUSE_PORT_TCP`: Server TCP port (default: `9000`)
- `CLICKHOUSE_PORT_HTTP`: Server HTTP port (default: `8123`)
Quick Install
# Install optional Python dependencies
pip3 install jinja2 termcolor
# Start ClickHouse server for testing
./build/programs/clickhouse server --config-file tests/config/config.xml &
# Run a specific test
python3 tests/clickhouse-test 01234_my_test
# Run all stateless tests
python3 tests/clickhouse-test
# Run integration tests via Praktika
python -m ci.praktika run "integration" --test <selectors>
Code Evidence
Python 3 shebang from `tests/clickhouse-test:1`:
#!/usr/bin/env python3
Optional jinja2 import from `tests/clickhouse-test:59`:
try:
import jinja2
except ImportError:
jinja2 = None
Optional termcolor import from `tests/clickhouse-test:52`:
try:
from termcolor import colored
except ImportError:
colored = None
Database name normalization from `.claude/instructions.md:33-36`:
The test runner creates a temporary database with a random name (e.g., test_abc123) for each test.
After test execution, the random database name is replaced with default in stdout/stderr files
before comparison with .reference.
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `Connection refused` | ClickHouse server not running | Start server: `clickhouse server --config tests/config/config.xml` |
| `ModuleNotFoundError: No module named 'jinja2'` | jinja2 not installed; template tests will be skipped | `pip3 install jinja2` |
| Test output mismatch with `.reference` | Using `${CLICKHOUSE_DATABASE}` in reference file | Hardcode `default` as database name in `.reference` files; test runner normalizes automatically |
| Test hangs or times out | Server process not responding | Check server logs; stop stale processes with `pgrep -f clickhouse` and `kill` |
Compatibility Notes
- Database naming: The test runner creates random database names and normalizes them to `default` in output. Reference files must use `default`, not variable substitution.
- Parallel execution: Tests run in parallel by default. Only add `no-parallel` tag when strictly necessary (e.g., tests that modify global state).
- Blacklist files: Tests incompatible with certain configurations are listed in blacklist files (`tests/async_insert_blacklist.txt`, `tests/parallel_replicas_blacklist.txt`).
- Integration tests: Run via `python -m ci.praktika run "integration" --test <selectors>` from the repository root.