Workflow:Unstructured IO Unstructured Performance Profiling
| Knowledge Sources | |
|---|---|
| Domains | Performance, Profiling, Developer_Tools |
| Last Updated | 2026-02-12 09:30 GMT |
Overview
Developer workflow for benchmarking and profiling the Unstructured partition pipeline using time profiling, memory profiling, and flame graph visualization to identify performance bottlenecks.
Description
This workflow describes the performance profiling and benchmarking process for the Unstructured partition engine. It provides three complementary profiling approaches: time profiling using cProfile to measure function-level execution time, memory profiling using memray to track memory allocations, and detailed runtime profiling using py-spy to generate speedscope-compatible flame graphs. The profiling infrastructure supports all four partitioning strategies (auto, fast, hi_res, ocr_only) and includes a standard benchmark document (War and Peace) for reproducible comparisons. Results are stored persistently and can be visualized using flamegraph, snakeviz, or speedscope tools.
Key capabilities:
- Three profiling modes: time (cProfile), memory (memray), runtime (py-spy)
- Support for all partitioning strategies
- Standard benchmark document for reproducible comparisons
- Multiple visualization options (flamegraph, snakeviz, speedscope, memray table/tree/summary)
- Docker support for profiling within containers
- Persistent results storage for comparison across runs
Usage
Execute this workflow when you need to measure the performance characteristics of the Unstructured partition pipeline, identify bottlenecks in document processing, compare performance across strategies, or validate that code changes have not introduced performance regressions. This is a developer-focused workflow used during optimization and release validation.
Execution Steps
Step 1: Environment_Preparation
Install the profiling dependencies (cProfile is built-in, memray and py-spy require separate installation). Prepare the test document by selecting from the included benchmark files or providing a custom document. The standard benchmark is War and Peace (1225 pages of text), with a 1-page excerpt available for quick tests. A warmup document is also included to prime the runtime before profiling.
Key considerations:
- Install profiling requirements from scripts/performance/requirements.txt
- Ensure flameprof and snakeviz are available for time profile visualization
- memray requires Linux or macOS (not available on Windows)
- py-spy requires elevated permissions on some systems
- Docker profiling mode is available via DOCKER_TEST environment variable
Step 2: Strategy_And_Document_Selection
Choose the partitioning strategy to profile (auto, fast, hi_res, ocr_only) and select the test document. Different strategies exercise different code paths: fast uses direct text extraction, hi_res loads and runs deep learning layout models, and ocr_only runs full OCR. The choice of document also affects which code paths are exercised (PDFs vs text files vs images).
Key considerations:
- Profile each strategy separately to understand their individual characteristics
- Use the full War and Peace document for realistic benchmarks
- Use the 1-page excerpt for quick iteration during optimization
- Custom documents can be provided for format-specific profiling
Step 3: Time_Profiling
Run the partition function under cProfile to capture function-level timing data. The profiler records the number of calls and cumulative time for every function in the call stack. Results are saved as .prof files that can be visualized as flame graphs using flameprof or interactively explored using snakeviz.
Key considerations:
- cProfile adds overhead; measured times are relative, not absolute
- Look for functions with high cumulative time and high call counts
- Use flame graphs to identify the deepest hot paths
- Use snakeviz for interactive drill-down into specific call stacks
Step 4: Memory_Profiling
Run the partition function under memray to track memory allocations. This captures every allocation and deallocation, identifying functions that allocate the most memory and peak memory usage. Results are saved as .bin files and can be viewed as flame graphs, tables, trees, summaries, or statistics.
Key considerations:
- Memory profiling has lower overhead than time profiling
- Look for unexpected memory spikes during processing
- The table view shows top allocators ranked by total allocation size
- The tree view shows the allocation call tree for understanding memory flow
- Compare memory profiles across strategies to understand memory-accuracy tradeoffs
Step 5: Runtime_Profiling
Run the partition function under py-spy to capture sampling-based runtime profiles. py-spy attaches to the running process and samples the call stack at regular intervals, producing speedscope-compatible output files. This approach has minimal overhead and captures native code execution alongside Python code.
Key considerations:
- py-spy uses sampling and has very low overhead compared to cProfile
- Output files can be viewed at speedscope.app for interactive exploration
- py-spy captures native C extension code that cProfile misses
- Useful for identifying bottlenecks in compiled libraries (numpy, PDF parsers, etc.)
Step 6: Results_Analysis
Analyze the collected profiles to identify performance bottlenecks and optimization opportunities. Compare results across strategies and document types. Store profiles persistently for regression tracking. Use the visualization tools to communicate findings and validate optimizations.
Key considerations:
- Results are stored in scripts/performance/profile_results/ for persistence
- Compare profiles before and after optimization to validate improvements
- Focus on the top cumulative-time functions for the highest impact optimizations
- Consider both time and memory when evaluating tradeoffs