Principle:Eventual Inc Daft Query Plan Inspection
| Knowledge Sources | |
|---|---|
| Domains | Data_Engineering, Query_Optimization |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
Technique for visualizing and understanding the logical and physical execution plans of a DataFrame query.
Description
Query plan inspection shows how Daft will execute a query, including optimizations like predicate pushdown and partition pruning. It reveals the sequence of operations (scans, filters, projections, joins, aggregations) and how the optimizer has rewritten them for efficient execution. This is essential for debugging performance issues, understanding why a query is slow, and verifying that expected optimizations are being applied.
Usage
Use query plan inspection when you need to understand or debug how a DataFrame query will be executed. This is particularly valuable when diagnosing slow queries, verifying that filters are being pushed down to the data source, or confirming that unnecessary columns are being pruned early in the plan.
Theoretical Basis
Query plan visualization operates at multiple levels of abstraction:
Plan Levels:
1. Unoptimized Logical Plan
- Direct translation of user operations
- No rewriting or optimization applied
2. Optimized Logical Plan
- After applying optimization rules:
* Predicate pushdown
* Projection pruning
* Constant folding
* Filter merging
3. Physical Plan
- Concrete execution operators
- Partition-level parallelism
- Shuffle and exchange operators
- Resource allocation decisions
Visualization Formats:
- ASCII tree: text-based tree representation
- Mermaid diagram: graphical flowchart rendering
Comparing the unoptimized and optimized plans reveals the transformations applied by the query optimizer, while the physical plan shows the actual execution strategy.