Principle:Fede1024 Rust rdkafka Latency Benchmarking
| Knowledge Sources | |
|---|---|
| Domains | Benchmarking, Performance, Latency_Measurement |
| Last Updated | 2026-02-07 19:00 GMT |
Overview
Principle of measuring end-to-end produce-consume latency through timestamp-embedded messages and HDR histogram recording, providing percentile-based performance characterization.
Description
Latency benchmarking for message queues involves measuring the time between a producer sending a message and a consumer receiving it. The standard approach embeds a wall-clock timestamp in the message payload at send time, then computes the delta at receive time. To account for JIT warmup, connection establishment, and consumer group rebalancing, measurements are collected in phases: a warm-up period (discarded), a recording period (captured in a histogram), and a reporting phase. HDR (High Dynamic Range) histograms are used because they can record values across a wide range with configurable precision while maintaining constant memory usage, unlike naive min/max/avg which lose distribution information.
Usage
Apply this principle when evaluating Kafka deployment performance, comparing runtime configurations, measuring the impact of serialization formats, or validating SLA compliance. The warm-up phase is critical for realistic results, as initial messages incur metadata fetch, connection setup, and consumer group join overhead that does not represent steady-state performance.
Theoretical Basis
Measurement Protocol:
Phase 1: Warm-up (e.g., 10 seconds)
- Producer sends timestamped messages continuously
- Consumer receives and discards measurements
- Purpose: Stabilize connections, fill caches, complete rebalancing
Phase 2: Recording (e.g., 10 seconds)
- latency = receive_time - send_time (wall-clock delta)
- Record each latency value in HDR histogram
- Histogram provides O(1) recording with configurable precision
Phase 3: Reporting
- Extract percentiles: p50, p90, p95, p99, p99.9
- Report mean, min, max
- p99 is typically the most operationally relevant metric
Key Considerations:
- Clock skew: Producer and consumer must run on the same machine for wall-clock comparison
- Histogram precision: HDR histograms use 3 significant digits by default
- Tail latency: p99 and p99.9 reveal worst-case behavior often hidden by averages
- Warm-up duration: Must exceed consumer group rebalance time (typically 5-30 seconds)