Principle:MaterializeInc Materialize Upgrade Validation

Knowledge Sources	misc/python/materialize/checks/actions.py, misc/python/materialize/checks/checks.py, misc/python/materialize/checks/executors.py
Domains	Upgrade Testing, Data Integrity, Regression Testing
Last Updated	2026-02-08

Overview

Post-upgrade data integrity verification asserts that all state created before an upgrade remains correct after the upgrade, enforcing the principle of regression-free upgrades.

Description

The Upgrade Validation principle establishes that an upgrade is correct if and only if all pre-upgrade state is fully preserved and queryable after the upgrade. This is the strongest practical guarantee a database system can provide: users should never lose data, views, connections, sinks, or any other catalog object as a result of a version upgrade.

The validation phase is distinguished from the initialization and manipulation phases by several key properties:

1. Post-hoc execution — Validation runs after the disruptive event (upgrade, restart, failover). It never runs before the event.
2. Read-only semantics — Validation queries the system state but does not modify it. This ensures that validation is safe to repeat.
3. Idempotent invocation — Many scenarios call Validate multiple times (e.g., after the upgrade and after an additional restart on the new version). The validation logic must produce the same result regardless of how many times it is called.
4. Comprehensive coverage — Validation checks span all Materialize object types: tables, views, materialized views, indexes, sources, sinks, connections, secrets, clusters, and replicas.
5. Version-aware assertions — Some checks need to adjust their expected output based on the base version (the version at which state was created) versus the current version. The Check.base_version and Check.current_version attributes support this.

The validation phase is the test oracle in the upgrade testing framework. If any validation check fails, it constitutes a regression: the upgrade has corrupted or lost state.

Usage

Apply this principle when:

• Writing a new upgrade check — the validate() method must assert everything that initialize() and manipulate() created.
• Debugging upgrade failures — a failing validation identifies exactly which object or query is broken.
• Designing upgrade scenarios — scenarios should include at least one Validate action after the upgrade, and ideally a second one after an additional restart to catch non-idempotent issues.
• Evaluating upgrade safety — the set of passing validation checks defines the contract of what the upgrade preserves.

Theoretical Basis

Upgrade validation implements the concept of a test oracle in formal testing theory. Given:

• A system state S before the upgrade.
• An upgrade operation U.
• A system state S' = U(S) after the upgrade.

The oracle function O(S, S') returns pass if and only if the observable behavior of S' is equivalent to the expected behavior given S. In practice, this means:

• All rows inserted before the upgrade are present after the upgrade.
• All materialized views return the same results.
• All sources and sinks continue to function.
• All catalog objects (connections, secrets, clusters) are accessible.

The framework distributes the oracle across many independent Check subclasses, each responsible for a specific feature area. This follows the decomposition principle: a complex oracle is easier to maintain when split into many small, focused oracles.

The idempotency requirement on validate() ensures that the oracle is a pure function of the system state — it does not depend on how many times it has been called or what state the validation framework itself is in. This property is formally verified by scenarios like RestartEnvironmentdClusterdStorage, which calls Validate twice in succession.

Validation Property	Formal Requirement	Practical Meaning
Completeness	Every state mutation in initialize()/manipulate() has a corresponding assertion in validate()	No untested state survives the upgrade
Soundness	validate() fails only when actual state differs from expected state	No false positives
Idempotency	validate(); validate() produces the same result as validate()	Safe to call multiple times

Related Pages

Implemented By

• Implementation:MaterializeInc_Materialize_Validate_Action