Implementation:MaterializeInc Materialize Zippy Test Framework

Knowledge Sources	MaterializeInc_Materialize
Domains	Testing, Chaos Engineering, Resilience
Last Updated	2026-02-08 00:00 GMT

Overview

The Zippy Test Framework is a randomized chaos testing engine that generates and executes weighted sequences of actions against Materialize to validate system resilience under disruptive conditions.

Description

Zippy implements a capability-based action selection system where each action declares its required capabilities (preconditions) and provided capabilities (postconditions). The framework randomly selects actions based on configurable weights from scenario definitions, enabling tests that interleave normal operations (data ingestion, view creation, query validation) with disruptive events (process kills, restarts, backup/restore). Scenarios are defined as classes that specify bootstrap actions, weighted action distributions, and finalization steps.

Usage

Use the Zippy framework when testing Materialize resilience against process failures, restarts, zero-downtime deployments, and other disruptive events. Scenarios cover Kafka sources, user tables, Debezium CDC, PostgreSQL CDC, MySQL CDC, SQL Server CDC, cluster replicas, and combinations thereof.

Code Reference

Source Location

Repository: MaterializeInc_Materialize
File: misc/python/materialize/zippy/framework.py
File: misc/python/materialize/zippy/scenarios.py

Signature

# framework.py - Core classes
class State:
    mz_service: str
    deploy_generation: int
    system_parameter_defaults: dict[str, str]

class Capability:
    name: str
    @classmethod
    def format_str(cls) -> str: ...

class Capabilities:
    def provides(self, capability: type[T]) -> bool: ...
    def get(self, capability: type[T]) -> list[T]: ...
    def get_free_capability_name(self, capability: type[T], max_objects: int) -> str | None: ...

class Action:
    @classmethod
    def requires(cls) -> set[type[Capability]] | list[set[type[Capability]]]: ...
    @classmethod
    def incompatible_with(cls) -> set[type[Capability]]: ...
    def withholds(self) -> set[type[Capability]]: ...
    def provides(self) -> list[Capability]: ...
    def run(self, c: Composition, state: State) -> None: ...

class ActionFactory:
    def new(self, capabilities: Capabilities) -> list[Action]: ...

class Test:
    def __init__(self, scenario: Scenario, actions: int, max_execution_time: timedelta) -> None: ...
    def run(self, c: Composition) -> None: ...

# scenarios.py - Scenario definitions
class Scenario:
    def bootstrap(self) -> list[ActionOrFactory]: ...
    def actions_with_weight(self) -> dict[ActionOrFactory, float]: ...
    def finalization(self) -> list[ActionOrFactory]: ...

Import

from materialize.zippy.framework import Action, ActionFactory, Capabilities, Capability, State, Test
from materialize.zippy.scenarios import Scenario, KafkaSources, UserTables, PostgresCdc

I/O Contract

Input	Type	Description
scenario	`Scenario`	Defines bootstrap, weighted actions, and finalization steps
actions	`int`	Number of random actions to generate in the test
max_execution_time	`timedelta`	Maximum wall-clock time for test execution

Output	Type	Description
test execution	side effects	Actions are executed against a live Materialize Composition; validation actions raise on failure

Scenario	Description	Key Actions
KafkaSources	Kafka source ingestion with disruptions	Ingest, CreateSource, MzStop, Mz0dtDeploy, KillClusterd
UserTables	Table DML with process kills and backup/restore	DML, CreateTable, MzStop, BackupAndRestore, KillClusterd
PostgresCdc	PostgreSQL CDC with storage disruptions	PostgresDML, CreatePostgresCdcTable, StoragedKill, PostgresRestart
MySqlCdc	MySQL CDC with storage disruptions	MySqlDML, CreateMySqlCdcTable, StoragedKill, MySqlRestart
SqlServerCdc	SQL Server CDC with storage disruptions	SqlServerDML, CreateSqlServerCdcTable, StoragedKill, SqlServerRestart
ClusterReplicas	Replica management under chaos	CreateReplica, DropReplica, KillClusterd, Ingest

Usage Examples

from datetime import timedelta
from materialize.zippy.framework import Test
from materialize.zippy.scenarios import KafkaSources

# Generate and run a 1000-action chaos test
scenario = KafkaSources()
test = Test(scenario=scenario, actions=1000, max_execution_time=timedelta(hours=1))
test.run(c=composition)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment