Heuristic:Risingwavelabs Risingwave LSM Compaction Tuning
| Knowledge Sources | |
|---|---|
| Domains | Storage, Optimization, Performance |
| Last Updated | 2026-02-09 08:00 GMT |
Overview
Hummock LSM-tree storage uses 6-level compaction with 512MB level base, 128MB minimum compaction size, 2GB maximum compaction size, and stalls writes when L0 reaches 300GB to prevent write amplification runaway.
Description
RisingWave's Hummock storage engine uses an LSM-tree (Log-Structured Merge-tree) design. Compaction is the process of merging and sorting SSTables across levels to optimize read performance. The default configuration defines critical thresholds: compaction triggers when L0 accumulates 12 SSTables (tier compact trigger), the level base starts at 512MB, each compaction job processes between 128MB and 2GB of data, and the tree has a maximum of 6 levels. A critical safety mechanism stops writes entirely when L0 grows beyond 300GB, preventing unbounded write amplification.
Usage
Apply this heuristic when tuning write-heavy workloads, diagnosing write stalls, investigating compaction lag, or configuring Hummock storage for specific hardware profiles. Understanding these thresholds helps operators balance write throughput against read latency and disk I/O.
The Insight (Rule of Thumb)
- Level Base: 512MB (
DEFAULT_MAX_BYTES_FOR_LEVEL_BASE) — first compaction level target size. - Compaction Range: 128MB min, 2GB max per compaction job.
- Tier Trigger: 12 L0 SSTables trigger compaction (
DEFAULT_TIER_COMPACT_TRIGGER_NUMBER = 12). - Max Levels: 6 levels in the LSM-tree.
- Write Stall: Writes stall at 300GB L0 size (
DEFAULT_LEVEL0_STOP_WRITE_THRESHOLD_MAX_SIZE = 300GB). - XOR16 Filter Limit: Bloom filters switch strategy above 256K entries.
- Trade-off: Lower compaction thresholds reduce read amplification but increase write amplification. Higher thresholds improve write throughput but degrade read performance.
Reasoning
LSM-tree performance is governed by the tension between write amplification (data rewritten during compaction), read amplification (number of levels consulted during reads), and space amplification (ratio of logical to physical data size). The default values represent a balanced configuration:
The 512MB level base ensures L1 is large enough to absorb typical write bursts without triggering excessive compaction. The 2GB max compaction size prevents individual compaction jobs from dominating I/O bandwidth. The 300GB L0 write stall threshold is a last-resort safety mechanism — in normal operation, compaction should keep L0 well below this limit.
The XOR16 bloom filter limit of 256K entries per filter is based on the observation that XOR16 filters have increasing false positive rates at very high key counts, and the construction algorithm becomes less efficient.
Code evidence from src/common/src/config/meta.rs lines 798-822:
const DEFAULT_MAX_COMPACTION_BYTES: u64 = 2 * GB;
const DEFAULT_MIN_COMPACTION_BYTES: u64 = 128 * MB;
const DEFAULT_MAX_BYTES_FOR_LEVEL_BASE: u64 = 512 * MB;
const DEFAULT_TIER_COMPACT_TRIGGER_NUMBER: u64 = 12;
const DEFAULT_MAX_LEVEL: u32 = 6;
const DEFAULT_LEVEL0_STOP_WRITE_THRESHOLD_MAX_SIZE: u64 = 300 * 1024 * MB;
Code evidence for XOR16 limit from src/common/src/config/meta.rs line 825:
pub const DEFAULT_MAX_KV_COUNT_FOR_XOR16: u64 = 256 * 1024;