Implementation:ClickHouse ClickHouse Lightweight Updates Stress
| Knowledge Sources | |
|---|---|
| Domains | Testing, Performance |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
A stress testing tool for ClickHouse's lightweight `UPDATE` feature that validates correctness under concurrent operations.
Description
This Python-based stress test validates ClickHouse's lightweight update functionality by running concurrent inserts, updates, and selects against various table engines (`MergeTree`, `SharedMergeTree`, `ReplicatedMergeTree`). It verifies correctness by replaying all operations on a reference `EmbeddedRocksDB` table and comparing final results. The test measures performance metrics including update throughput, select query slowdown with patches applied, and the number of patch parts generated.
The implementation uses multithreading to simulate realistic concurrent workloads, tracks all queries via unique IDs, and validates sequential consistency by replaying operations in the correct order based on block allocation timestamps.
Usage
Use this tool when you need to:
- Test lightweight update functionality under stress conditions
- Validate correctness of concurrent updates and reads
- Measure performance impact of patch parts on queries
- Test different table engines with lightweight updates enabled
Code Reference
Source Location
- Repository: ClickHouse
- File: utils/lightweight_updates_stress.py
- Lines: 1-460
Signature
def run_query(query, input_data=None, query_id=None)
def run_insert()
def run_update(update_mode)
def run_select(apply_patches)
def flush_logs()
def print_short_stats()
def print_full_stats()
Import
#!/usr/bin/env python3
./lightweight_updates_stress.py [--timeout 10] [--threads 8] [--engine SharedMergeTree]
I/O Contract
| Input | Type | Description |
|---|---|---|
| --timeout | float | Test duration in seconds (default: 10) |
| --threads | int | Number of concurrent threads (default: 8) |
| --columns | int | Number of columns in test table (default: 10) |
| --scale | int | Range of random values (default: 100) |
| --engine | str | Table engine: MergeTree, SharedMergeTree, ReplicatedMergeTree |
| --with-mutations | bool | Enable heavy mutations in addition to lightweight updates |
| Output | Type | Description |
|---|---|---|
| Statistics | stdout | Updates RPS, select slowdown, part counts |
| Exit code | int | 0 if data matches, 1 if validation fails |
Usage Examples
# Basic stress test with default settings
./lightweight_updates_stress.py
# Test with specific engine and longer timeout
./lightweight_updates_stress.py --engine ReplicatedMergeTree --timeout 60
# Test with more threads and columns
./lightweight_updates_stress.py --threads 16 --columns 20
# Test with heavy mutations enabled
./lightweight_updates_stress.py --with-mutations --timeout 30
# Test against cloud instance
./lightweight_updates_stress.py --host https://example.clickhouse.cloud --user default --password secret
# Verbose output with detailed statistics
./lightweight_updates_stress.py --verbose --timeout 20
# Skip final correctness check (faster)
./lightweight_updates_stress.py --skip-check --timeout 10