Principle:Spcl Graph of thoughts Result Serialization

Field	Value
Pattern Name	Result_Serialization
Type	Principle
Repository	spcl/graph-of-thoughts
Domains	Serialization, Evaluation
Related Implementations	Implementation:Spcl_Graph_of_thoughts_Controller_Output_Graph

Overview

Pattern for serializing the complete state of a Graph Reasoning execution to JSON for analysis, debugging, and benchmarking.

Description

After Controller.run() completes, the full graph state can be serialized to a JSON file. The serialization captures the complete execution trace of every operation in the Graph of Operations, providing a detailed record of how the reasoning process unfolded. This includes:

• Operation type: The name of each operation (e.g., generate, score, keep_best_n, aggregate, selector, ground_truth_evaluator).
• Thought states: The full state dictionary of every thought produced by each operation. This captures the intermediate and final solutions at each stage.
• Scores: If thoughts were scored (by a Score operation), the serialized output includes both whether each thought was scored and the score values.
• Validity: If thoughts were validated (by a ValidateAndImprove operation), the output includes whether each thought was validated and whether it was found to be valid.
• Ground truth results: If thoughts were evaluated against a ground truth (by a GroundTruth operation), the output records whether the comparison was performed and whether the problem was solved correctly.
• Cost tracking: The final entry in the JSON array contains LLM usage statistics: total prompt tokens, completion tokens, and monetary cost.

The serialized output is an ordered JSON array. Each element corresponds to one operation in the graph, serialized in execution order. The final element is a cost summary dictionary. This structure makes it straightforward to reconstruct the full execution trace or to extract specific metrics (e.g., final accuracy, total cost) for benchmarking.

JSON Structure

The output JSON follows this schema:

[
  {
    "operation": "generate",
    "thoughts": [ { "original": "...", "current": "...", ... }, ... ]
  },
  {
    "operation": "score",
    "thoughts": [ ... ],
    "scored": [true, true, ...],
    "scores": [3.0, 1.0, ...]
  },
  {
    "operation": "keep_best_n",
    "thoughts": [ ... ]
  },
  {
    "operation": "aggregate",
    "thoughts": [ ... ]
  },
  {
    "operation": "validate_and_improve",
    "thoughts": [ ... ],
    "validated": [true, true, ...],
    "validity": [true, false, ...]
  },
  {
    "operation": "ground_truth_evaluator",
    "thoughts": [ ... ],
    "compared_to_ground_truth": [true],
    "problem_solved": [true]
  },
  {
    "prompt_tokens": 12345,
    "completion_tokens": 6789,
    "cost": 0.0567
  }
]

Fields like scored, scores, validated, validity, compared_to_ground_truth, and problem_solved are only present on operations whose thoughts have the corresponding attributes set (i.e., they are conditionally included based on whether any thought in the operation has that attribute).

Usage

Use this pattern after every GoT execution to persist results for later analysis or comparison. The typical workflow is:

1. Build a GraphOfOperations topology.
2. Create a Controller with the topology, language model, prompter, parser, and problem parameters.
3. Call controller.run() to execute the reasoning graph.
4. Call controller.output_graph(path) to serialize the results to a JSON file.
5. Use the JSON files for offline analysis, cross-method comparison, or visualization.

from graph_of_thoughts.controller import Controller

# After building and running the controller...
ctrl = Controller(lm, graph, prompter, parser, problem_params)
ctrl.run()

# Serialize the complete execution state to JSON
ctrl.output_graph("results/got/sample_42.json")

This pattern is used consistently across all GoT benchmark examples (sorting, keyword counting, document merging) to record results for each sample and method combination, enabling systematic comparison of different reasoning topologies.

Related

• Implementation:Spcl_Graph_of_thoughts_Controller_Output_Graph