Implementation:Ucbepic Docetl FastShouldOptimize
| Knowledge Sources | |
|---|---|
| Domains | Data_Processing, Optimization |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
Concrete tool for quickly determining whether a map operation should be decomposed provided by DocETL.
Description
The FastShouldOptimizeAnalyzer class provides a lightweight alternative to the full MapOptimizer/ReduceOptimizer/JoinOptimizer flow for determining if an operation warrants decomposition. Instead of running the operation on sample data and using complex evaluation logic, it reads cached outputs from intermediate files and makes a single LLM judgment call. The analyzer dynamically calculates how many output samples fit in the LLM context window, builds a structured analysis prompt, and returns a structured assessment with rationale and suggested improvements.
Usage
Use FastShouldOptimizeAnalyzer when you want to quickly assess whether a map operation would benefit from optimization without incurring the full cost of running the optimizer pipeline. It is ideal for pre-screening operations after an initial pipeline run has produced intermediate output files, allowing you to decide which operations deserve the more expensive full optimization pass.
Code Reference
Source Location
- Repository: Ucbepic_Docetl
- File: docetl/optimizers/fast_should_optimize.py
- Lines: 1-337
Signature
class FastShouldOptimizeAnalyzer:
def __init__(
self,
intermediate_dir: str,
optimizer_model: str = "gpt-5.1",
litellm_kwargs: dict[str, Any] | None = None,
) -> None: ...
def load_operation_data(self, step_name: str, op_name: str) -> list[dict[str, Any]]: ...
def find_previous_operation(self, operations: list[dict[str, Any]], op_name: str) -> str | None: ...
def get_max_context_tokens(self) -> int: ...
def calculate_samples_that_fit(self, op_config: dict[str, Any], outputs: list[dict[str, Any]]) -> list[dict[str, Any]]: ...
def build_analysis_prompt(self, op_config: dict[str, Any], samples: list[dict[str, Any]]) -> tuple[str, str]: ...
def analyze(self, op_config: dict[str, Any], step_name: str, op_name: str) -> tuple[str, list[dict[str, Any]], int, float]: ...
Import
from docetl.optimizers.fast_should_optimize import FastShouldOptimizeAnalyzer
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| intermediate_dir | str | Yes | Path to the directory containing intermediate outputs |
| optimizer_model | str | No | LLM model to use for analysis (default: "gpt-5.1") |
| litellm_kwargs | dict[str, Any] or None | No | Additional kwargs to pass to litellm.completion |
| op_config | dict[str, Any] | Yes | The operation configuration dictionary (for analyze method) |
| step_name | str | Yes | Name of the pipeline step (for analyze method) |
| op_name | str | Yes | Name of the operation (for analyze method) |
Outputs
| Name | Type | Description |
|---|---|---|
| rationale | str | Empty string if no optimization needed; explanation with suggested improvements if optimization recommended |
| output_samples | list[dict[str, Any]] | The samples that were analyzed |
| num_docs_analyzed | int | Number of documents that fit in the LLM prompt |
| cost | float | LLM API cost in USD |
Usage Examples
from docetl.optimizers.fast_should_optimize import FastShouldOptimizeAnalyzer
analyzer = FastShouldOptimizeAnalyzer(
intermediate_dir="/path/to/pipeline/intermediate",
optimizer_model="gpt-5.1",
)
op_config = {
"name": "extract_entities",
"type": "map",
"prompt": "Extract all named entities from {{ input.text }}",
"output": {"schema": {"entities": "list[string]"}},
}
rationale, samples, num_docs, cost = analyzer.analyze(
op_config=op_config,
step_name="extraction_step",
op_name="extract_entities",
)
if rationale:
print(f"Optimization recommended: {rationale}")
else:
print("No optimization needed.")
print(f"Analyzed {num_docs} documents, cost: ${cost:.4f}")