Implementation:Ucbepic Docetl Directive ReduceChaining
| Knowledge Sources | |
|---|---|
| Domains | Pipeline_Optimization, LLM_Operations |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
Concrete tool for inserting a Map extraction step before a Reduce operation to pre-process documents provided by the DocETL reasoning optimizer.
Description
The ReduceChainingDirective class transforms a reduce operation that processes long documents by inserting a Map step that extracts or processes relevant information from each document first, then modifying the reduce prompt to work with the processed results instead of full document content. This makes the reduce step more efficient and focused by operating on pre-extracted data.
Usage
The MOAR agent applies this directive when a reduce operation iterates through long documents to extract specific information (e.g., locations, entities, themes) that could be pre-extracted per document. The target operator must be a reduce operator.
Code Reference
Source Location
- Repository: Ucbepic_Docetl
- File: docetl/reasoning_optimizer/directives/reduce_chaining.py
- Lines: 1-304
Signature
class ReduceChainingDirective(Directive):
name = "reduce_chaining"
description = "Transform a reduce operation by inserting a Map step to pre-extract information from each document."
def check_applicability(self, ...) -> Tuple[bool, str]: ...
def apply(self, ...) -> Tuple[List[Dict], List[Dict], str, dict]: ...
Import
from docetl.reasoning_optimizer.directives.reduce_chaining import ReduceChainingDirective
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| op_config | Dict | Yes | Operation configuration to transform |
| pipeline_ops | List[Dict] | Yes | Full pipeline operations list |
| op_idx | int | Yes | Index of target operation |
| dataset_descriptions | Dict | Yes | Dataset schema descriptions |
Outputs
| Name | Type | Description |
|---|---|---|
| new_ops | List[Dict] | Transformed operation configs |
| new_steps | List[Dict] | Updated pipeline steps |
| explanation | str | Human-readable description of changes |
| metadata | dict | Additional metadata about the transformation |
Usage Examples
# Directives are typically invoked by the MOAR agent automatically
# Example of manual invocation:
from docetl.reasoning_optimizer.directives.reduce_chaining import ReduceChainingDirective
directive = ReduceChainingDirective()
applicable, reason = directive.check_applicability(op_config, pipeline_ops, op_idx, dataset_descriptions)
if applicable:
new_ops, new_steps, explanation, metadata = directive.apply(op_config, pipeline_ops, op_idx, dataset_descriptions)