Implementation:Ucbepic Docetl Directive ArbitraryRewrite
| Knowledge Sources | |
|---|---|
| Domains | Pipeline_Optimization, LLM_Operations |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
Concrete tool for arbitrary search/replace edits on the pipeline JSON provided by the DocETL reasoning optimizer.
Description
The ArbitraryRewriteDirective class allows the MOAR agent to make free-form edits to the pipeline using search/replace operations on the JSON representation. It serves as a catch-all directive for optimizations that do not fit into other specific directive patterns, enabling adding, modifying, removing, or replacing operations to optimize for cost, accuracy, or both.
Usage
The MOAR agent applies this directive when it identifies obvious optimizations that can make the pipeline cheaper or more accurate but they do not fit into existing directive patterns. Used for complex multi-operation changes, reordering operations, consolidating redundant operations, or making systematic improvements across the pipeline.
Code Reference
Source Location
- Repository: Ucbepic_Docetl
- File: docetl/reasoning_optimizer/directives/arbitrary_rewrite.py
- Lines: 1-361
Signature
class ArbitraryRewriteDirective(Directive):
name = "arbitrary_rewrite"
description = "Allows the agent to make arbitrary edits to the pipeline using search/replace operations on the JSON representation."
def check_applicability(self, ...) -> Tuple[bool, str]: ...
def apply(self, ...) -> Tuple[List[Dict], List[Dict], str, dict]: ...
Import
from docetl.reasoning_optimizer.directives.arbitrary_rewrite import ArbitraryRewriteDirective
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| op_config | Dict | Yes | Operation configuration to transform |
| pipeline_ops | List[Dict] | Yes | Full pipeline operations list |
| op_idx | int | Yes | Index of target operation |
| dataset_descriptions | Dict | Yes | Dataset schema descriptions |
Outputs
| Name | Type | Description |
|---|---|---|
| new_ops | List[Dict] | Transformed operation configs |
| new_steps | List[Dict] | Updated pipeline steps |
| explanation | str | Human-readable description of changes |
| metadata | dict | Additional metadata about the transformation |
Usage Examples
# Directives are typically invoked by the MOAR agent automatically
# Example of manual invocation:
from docetl.reasoning_optimizer.directives.arbitrary_rewrite import ArbitraryRewriteDirective
directive = ArbitraryRewriteDirective()
applicable, reason = directive.check_applicability(op_config, pipeline_ops, op_idx, dataset_descriptions)
if applicable:
new_ops, new_steps, explanation, metadata = directive.apply(op_config, pipeline_ops, op_idx, dataset_descriptions)