Workflow:Diagram of thought Diagram of thought DoT Prompt Customization
| Knowledge Sources | |
|---|---|
| Domains | Prompt_Engineering, LLM_Reasoning, Task_Adaptation |
| Last Updated | 2024-09-15 12:00 GMT |
Overview
End-to-end process for adapting the Diagram of Thought prompt to specific task domains and rigor requirements, from strict formal reasoning to flexible creative exploration.
Description
This workflow guides users through customizing the DoT prompt for their specific use case. The base DoT framework provides two key customization axes: the rigor level of the critic role (strict for mathematics and logic versus flexible for creative and open-ended tasks) and the typed protocol inclusion (full typed records for auditability versus simplified role-only mode for basic use). Users select the appropriate configuration, adapt the prompt template with domain-specific instructions, and validate the customized prompt on representative examples. The result is a tailored DoT prompt optimized for the target domain.
Usage
Execute this workflow when the standard DoT iterative reasoning prompt needs adaptation for a specific domain or task type. Common triggers include: needing stricter validation for safety-critical domains (finance, medicine, code verification), wanting softer critique for creative or exploratory tasks (brainstorming, writing, design), or requiring domain-specific terminology and evaluation criteria in the proposer and critic roles.
Execution Steps
Step 1: Define Task Requirements
Characterize the target task along two dimensions. First, determine the rigor level: strict tasks (mathematics, formal logic, code correctness) require the critic to always emit explicit validation/invalidation decisions, while flexible tasks (creative writing, brainstorming, open-ended analysis) allow softer critiques with suggestions rather than hard rejections. Second, determine whether the typed serialization protocol is needed: include @node, @edge, and @status records for tasks requiring auditability and formal verification, or omit them for simpler use cases.
Key considerations:
- Strict mode: critic must always emit @status records with validated/invalidated marks
- Flexible mode: critic provides feedback but may use softer language without hard invalidation
- Typed protocol is essential for post-hoc trace extraction and formal guarantees
- Without typed records, the reasoning is still structured but not machine-parseable
Step 2: Select Base Prompt Template
Choose the appropriate starting template from the repository. The full iterative-reasoner prompt (from prompts/iterative-reasoner.md) provides the complete role definitions with XML tags and process flow instructions. The minimal prompt template (from the README) provides a compact version with typed records and role tokens. Select based on whether you need the detailed behavioral instructions or a streamlined version.
Key considerations:
- Full prompt: detailed role responsibilities, formatting guidelines, example interaction
- Minimal prompt: compact, includes typed record syntax, suitable for capable models
- The full prompt is better for models that need explicit behavioral guidance
- The minimal prompt works well with frontier models that follow structured instructions
Step 3: Customize Role Instructions
Modify the role-specific instructions to align with the target domain. For the proposer, add domain-specific guidance on what constitutes a valid reasoning step (e.g., "cite relevant legal precedents" for legal analysis, or "include mathematical notation" for proofs). For the critic, calibrate the evaluation criteria and rigor level. For the summarizer, specify the expected output format and synthesis requirements.
What happens:
- Proposer instructions are augmented with domain-specific step requirements
- Critic evaluation criteria are tuned to the appropriate rigor level
- Summarizer output format is aligned with the expected deliverable
- Any domain-specific constraints or terminology are injected into role definitions
Step 4: Configure Typed Protocol
If the typed serialization protocol is needed, ensure the @node, @edge, and @status record formats are included in the prompt with any domain-specific adaptations. If not needed, simplify the prompt to use only XML role tags without typed records. For strict mode, add explicit instructions requiring the critic to always emit @status records. For flexible mode, make @status records optional.
Key considerations:
- Strict mode should mandate: "You MUST emit a @status record for every proposition"
- Flexible mode can state: "Optionally emit @status records when a clear judgment is possible"
- Edge kind types (use, critique, refine) can be extended for domain-specific relationships
- The acyclicity constraint (src < dst) must always be preserved if typed records are used
Step 5: Validate on Representative Examples
Test the customized prompt on 2 to 3 representative examples from the target domain. Verify that the model correctly follows the role alternation pattern, the critic provides appropriate-level feedback, the typed records (if included) are well-formed, and the summarizer produces useful conclusions. Iterate on the prompt if the model deviates from expected behavior.
What happens:
- Run the customized prompt on representative problems
- Check that role alternation follows the expected pattern
- Verify typed records are syntactically correct and the graph is acyclic
- Assess whether critic rigor matches the target level
- Refine prompt wording if model behavior deviates from expectations