Principle:Apache Druid Partitioning Configuration
| Knowledge Sources | |
|---|---|
| Domains | Data_Ingestion, Storage_Optimization |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
A storage optimization principle that controls how ingested data is divided into segments based on time intervals and secondary partitioning strategies.
Description
Partitioning Configuration determines how Druid distributes ingested data across segments, directly impacting query performance, storage efficiency, and cluster resource utilization. Partitioning operates at two levels:
- Primary partitioning (segmentGranularity): Divides data into time-based chunks (HOUR, DAY, WEEK, MONTH, YEAR, ALL). Each time chunk becomes a separate segment.
- Secondary partitioning (partitionsSpec): Further divides each time chunk into multiple segments based on row count or value ranges.
Four secondary partitioning strategies are available:
- Dynamic: Segments are created when a row count threshold is reached (simplest, single-phase)
- Hashed: Rows are distributed across a fixed number of segments by hash (uniform distribution)
- Single_dim: Rows are range-partitioned on a single dimension (optimal for high-cardinality filtering)
- Range: Rows are range-partitioned on multiple dimensions
Usage
Use this principle after schema definition to optimize storage layout for your query patterns. Proper partitioning is essential for production deployments — incorrect partitioning leads to either too many small segments (overhead) or too few large segments (slow queries).
Theoretical Basis
Partitioning follows a two-level segmentation model:
Data → Time Chunks (segmentGranularity)
→ Segments per chunk (partitionsSpec)
segmentGranularity: 'HOUR' | 'DAY' | 'WEEK' | 'MONTH' | 'YEAR' | 'ALL'
partitionsSpec.type:
'dynamic' → targetRowsPerSegment (default: 5,000,000)
'hashed' → numShards, partitionDimensions
'single_dim' → targetRowsPerSegment, partitionDimension
'range' → targetRowsPerSegment, partitionDimensions