Implementation:Eventual Inc Daft DataFrame Collect
| Knowledge Sources | |
|---|---|
| Domains | Data_Engineering, Query_Optimization |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
Concrete tool for triggering execution of a lazy DataFrame and materializing all results into memory provided by the Daft library.
Description
The collect method on Daft's DataFrame class triggers execution of the entire lazy query plan and materializes all results into memory. This is a blocking call that waits for all partitions to complete execution. After collection, the DataFrame holds the concrete result data and can be used for further operations without re-execution. A configurable num_preview_rows parameter controls how many rows are displayed when the DataFrame is printed.
Usage
Use df.collect() when you need the full materialized results of a DataFrame computation. This is required before accessing row values, passing data to non-Daft libraries, or when you want to cache results for repeated access.
Code Reference
Source Location
- Repository: Daft
- File:
daft/dataframe/dataframe.py - Lines: L4195-4233
Signature
def collect(self, num_preview_rows: int | None = 8) -> DataFrame
Import
import daft
# Method on DataFrame - no separate import needed
df = df.collect()
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| num_preview_rows | int or None | No | Number of rows to preview when printing. Defaults to 8. Set to None to preview all rows. |
Outputs
| Name | Type | Description |
|---|---|---|
| return | DataFrame | The same DataFrame, now with all results materialized in memory |
Usage Examples
Basic Usage
import daft
# Build a lazy query plan
df = daft.from_pydict({"x": [1, 2, 3], "y": [4, 5, 6]})
df = df.where(daft.col("x") > 1)
# Trigger execution and materialize results
df = df.collect()
df.show()
# Output:
# x: [2, 3]
# y: [5, 6]
Custom Preview Rows
import daft
df = daft.from_pydict({"x": list(range(100))})
# Collect with custom preview row count
df = df.collect(num_preview_rows=3)