Heuristic:Onnx Onnx Shape Inference Limitations
| Knowledge Sources | |
|---|---|
| Domains | Model_Validation, Debugging |
| Last Updated | 2026-02-10 02:00 GMT |
Overview
Shape inference in ONNX is not guaranteed to be complete; conflicts indicate model errors, not inference bugs.
Description
The ONNX shape inference engine attempts to propagate type and shape information through the computation graph. However, it has inherent limitations: it cannot always infer shapes for all operators, especially those with data-dependent output shapes (e.g., NonZero, Compress), dynamic shapes, or custom operators. The engine operates in a best-effort mode by default, silently stopping when it encounters an operator it cannot handle. In strict mode, it raises errors instead of silently skipping. If inferred shapes conflict with shapes already present in the graph, the existing values are considered invalid.
Usage
Use this heuristic when:
- Shape inference returns incomplete results (some value_info entries lack shape information)
- Inferred shapes conflict with shapes already specified in the graph
- Using custom operators that the standard shape inference engine does not know about
- Debugging model validation failures where shapes appear inconsistent
- Working with data-dependent shapes (NonZero, Where with dynamic conditions, etc.)
The Insight (Rule of Thumb)
- Action: Do not rely on shape inference to validate all shapes. Use `strict_mode=True` for thorough checking but expect it to fail on models with custom or experimental operators. When shapes conflict, trust the inference result and fix the model.
- Value: `check_type=True` enables type-equality validation; `strict_mode=True` enables error throwing instead of silent stopping; `data_prop=True` enables data propagation for statically known tensor values.
- Trade-off: Strict mode catches more issues but may reject valid models that use custom operators. Data propagation improves inference but adds computation cost.
Reasoning
Shape inference operates on the operator schema definitions. When an operator schema does not define a shape inference function, or when the inference function cannot determine the output shape (e.g., data-dependent shapes), the engine simply skips that node. This is by design: the ONNX spec explicitly states "Shape inference is not guaranteed to be complete."
The conflict resolution rule is important: if shape inference produces a shape that disagrees with a shape already specified in the graph, this means the model author made an error. The shape inference engine is considered correct because it follows the operator spec.
For models exceeding 2GB, use `infer_shapes_path()` instead of `infer_shapes()` to avoid loading the entire model into memory.
Code evidence from `onnx/shape_inference.py:5-6`:
"""onnx shape inference. Shape inference is not guaranteed to be
complete.
"""
Conflict handling from `onnx/shape_inference.py:42-44`:
# If the inferred values conflict with values already provided in the
# graph, that means that the provided values are invalid (or there is a
# bug in shape inference), and the result is unspecified.