Principle:MaterializeInc Materialize Upgrade Validation
| Knowledge Sources | misc/python/materialize/checks/actions.py, misc/python/materialize/checks/checks.py, misc/python/materialize/checks/executors.py |
|---|---|
| Domains | Upgrade Testing, Data Integrity, Regression Testing |
| Last Updated | 2026-02-08 |
Overview
Post-upgrade data integrity verification asserts that all state created before an upgrade remains correct after the upgrade, enforcing the principle of regression-free upgrades.
Description
The Upgrade Validation principle establishes that an upgrade is correct if and only if all pre-upgrade state is fully preserved and queryable after the upgrade. This is the strongest practical guarantee a database system can provide: users should never lose data, views, connections, sinks, or any other catalog object as a result of a version upgrade.
The validation phase is distinguished from the initialization and manipulation phases by several key properties:
- Post-hoc execution — Validation runs after the disruptive event (upgrade, restart, failover). It never runs before the event.
- Read-only semantics — Validation queries the system state but does not modify it. This ensures that validation is safe to repeat.
- Idempotent invocation — Many scenarios call
Validatemultiple times (e.g., after the upgrade and after an additional restart on the new version). The validation logic must produce the same result regardless of how many times it is called. - Comprehensive coverage — Validation checks span all Materialize object types: tables, views, materialized views, indexes, sources, sinks, connections, secrets, clusters, and replicas.
- Version-aware assertions — Some checks need to adjust their expected output based on the base version (the version at which state was created) versus the current version. The
Check.base_versionandCheck.current_versionattributes support this.
The validation phase is the test oracle in the upgrade testing framework. If any validation check fails, it constitutes a regression: the upgrade has corrupted or lost state.
Usage
Apply this principle when:
- Writing a new upgrade check — the
validate()method must assert everything thatinitialize()andmanipulate()created. - Debugging upgrade failures — a failing validation identifies exactly which object or query is broken.
- Designing upgrade scenarios — scenarios should include at least one
Validateaction after the upgrade, and ideally a second one after an additional restart to catch non-idempotent issues. - Evaluating upgrade safety — the set of passing validation checks defines the contract of what the upgrade preserves.
Theoretical Basis
Upgrade validation implements the concept of a test oracle in formal testing theory. Given:
- A system state S before the upgrade.
- An upgrade operation U.
- A system state S' = U(S) after the upgrade.
The oracle function O(S, S') returns pass if and only if the observable behavior of S' is equivalent to the expected behavior given S. In practice, this means:
- All rows inserted before the upgrade are present after the upgrade.
- All materialized views return the same results.
- All sources and sinks continue to function.
- All catalog objects (connections, secrets, clusters) are accessible.
The framework distributes the oracle across many independent Check subclasses, each responsible for a specific feature area. This follows the decomposition principle: a complex oracle is easier to maintain when split into many small, focused oracles.
The idempotency requirement on validate() ensures that the oracle is a pure function of the system state — it does not depend on how many times it has been called or what state the validation framework itself is in. This property is formally verified by scenarios like RestartEnvironmentdClusterdStorage, which calls Validate twice in succession.
| Validation Property | Formal Requirement | Practical Meaning |
|---|---|---|
| Completeness | Every state mutation in initialize()/manipulate() has a corresponding assertion in validate() |
No untested state survives the upgrade |
| Soundness | validate() fails only when actual state differs from expected state |
No false positives |
| Idempotency | validate(); validate() produces the same result as validate() |
Safe to call multiple times |