Principle:MaterializeInc Materialize Composition Service Definition

Knowledge Sources	misc/python/materialize/mzcompose/service.py, misc/python/materialize/mzcompose/services/materialized.py, misc/python/materialize/mzcompose/services/kafka.py, misc/python/materialize/mzcompose/services/schema_registry.py
Domains	Integration Testing, Service Composition, Distributed Systems
Last Updated	2026-02-08

Overview

Declarative service composition defines distributed system topologies for integration testing using typed service configurations that map directly to Docker Compose service definitions.

Description

In a distributed system like Materialize, integration tests must orchestrate multiple cooperating services: the Materialize database itself, Kafka brokers, schema registries, metadata stores, blob stores, and more. The Composition Service Definition principle provides a typed, declarative approach to specifying these service topologies in Python rather than raw YAML.

Each service is modeled as a Python class that inherits from a base Service class. The base class pairs a name (a string identifier) with a config (a typed dictionary conforming to the ServiceConfig TypedDict). The ServiceConfig TypedDict mirrors the Docker Compose service specification, supporting fields such as image, ports, environment, depends_on, healthcheck, volumes, command, and more.

Specialized subclasses such as Materialized, Kafka, and SchemaRegistry encode domain-specific defaults and dependency wiring. For example, the Materialized service automatically configures environment variables for telemetry suppression, soft assertions, cluster replica sizing, and metadata store connectivity. The Kafka service declares a health-checked dependency on ZooKeeper. The SchemaRegistry service declares a health-checked dependency on its Kafka brokers.

This pattern yields several advantages:

• Type safety: ServiceConfig is a TypedDict with named, typed fields, catching misconfiguration at development time.
• Composability: Services can be instantiated with different parameters and combined freely.
• Encapsulation: Domain-specific defaults (e.g., Materialize's MZ_NO_TELEMETRY=1) are encoded once in the service class, not repeated in every test.
• Dependency declaration: The depends_on field with ServiceDependency typed conditions (service_healthy, service_started) ensures correct startup ordering.

Usage

Use this principle when:

• Defining the set of services required for an integration test scenario.
• Creating a new service wrapper for a third-party dependency (e.g., a new database, message broker, or external tool).
• Customizing the configuration of an existing service for a specific test case (e.g., changing the Kafka replication factor or enabling unsafe mode in Materialize).
• Specifying inter-service dependencies and health check requirements.

Theoretical Basis

The principle draws on declarative infrastructure specification, where the desired state of a system is described rather than the procedural steps to achieve it. This is the same paradigm used by Docker Compose, Kubernetes manifests, and Terraform configurations.

The key theoretical elements are:

1. Typed configuration schemas: Using TypedDict to model configuration ensures that the configuration space is well-defined and that invalid configurations are caught statically. The ServiceConfig TypedDict mirrors the Docker Compose service schema, providing fields like image, mzbuild, ports, environment, depends_on, healthcheck, volumes, command, networks, and deploy.
2. Dependency graph declaration: The depends_on field forms a directed acyclic graph (DAG) of service dependencies with condition predicates (service_healthy or service_started). Docker Compose uses this DAG to determine startup order and readiness gating.
3. Template method pattern: Each service subclass overrides the constructor to build its ServiceConfig dictionary with domain-appropriate defaults, then delegates to the base Service.__init__. This is a form of the template method design pattern where the invariant structure (name + config) is fixed by the base class while the variant details are supplied by subclasses.
4. Health check contracts: Each service declares its own health check command (e.g., curl -f localhost:6878/api/readyz for Materialized, nc -z localhost 9092 for Kafka), establishing a contract for when the service is considered ready to accept traffic.

Related Pages

Implemented By

• Implementation:MaterializeInc_Materialize_Service_Init