Environment:MaterializeInc Materialize Docker Compose Runtime

Knowledge Sources	MaterializeInc/materialize Docker Compose
Domains	Infrastructure, Testing
Last Updated	2026-02-08 21:00 GMT

Overview

Docker and Docker Compose runtime for mzcompose-based integration tests, providing container orchestration for Materialize, Kafka, PostgreSQL, MySQL, and other service dependencies.

Description

This environment provides the container runtime required by the mzcompose test orchestration system. It wraps Docker Compose to manage multi-service test topologies, building custom Docker images via the mzbuild system and pulling third-party images from Docker Hub and GHCR. The environment requires Docker Engine with buildx support for multi-architecture builds, and uses content-addressed fingerprinting to cache and reuse previously built images.

Usage

Use this environment for running any mzcompose-based integration test, upgrade test, or benchmark. It is the mandatory prerequisite for Composition_Run, Composition_Up_Override, Service_Init, ResolvedImage_Build, and all Docker image build implementations.

System Requirements

Category	Requirement	Notes
OS	Linux (x86_64 or aarch64) or macOS	Docker Desktop or Docker Engine
Hardware	Varies by test	Some tests require 16+ GB RAM for multi-container topologies
Disk	50+ GB SSD	Docker images and build cache consume significant space
Software	Docker Engine 20.10+	Buildx plugin required for multi-arch builds

Dependencies

System Packages

`docker` (Docker Engine with CLI)
`docker compose` or `docker-compose` (Compose V2 preferred)
`docker buildx` (multi-architecture build support)
`git` (for mzbuild fingerprinting)

Python Packages

`psycopg` >= 3.0 (PostgreSQL driver for Materialize SQL connections)
`sqlparse` (SQL parsing for test queries)
`pyyaml` (YAML parsing for compose files)
`requests` (HTTP client for Docker registry API)

Credentials

The following environment variables may be required:

`DOCKERHUB_USERNAME`: DockerHub username for authenticated pulls (optional, avoids rate limits)
`DOCKERHUB_ACCESS_TOKEN`: DockerHub access token
`MZ_GHCR`: Set to `"1"` to use GitHub Container Registry instead of Docker Hub (default `"1"` in CI, `"0"` locally)

Quick Install

# Install Docker Engine and Compose
# Then run an mzcompose test:
bin/mzcompose --find <composition-name> run default

Code Evidence

Docker image listing from `mzbuild.py:237-243`:

def docker_images() -> set[str]:
    """List the Docker images available on the local machine."""
    return set(
        spawn.capture(["docker", "images", "--format", "{{.Repository}}:{{.Tag}}"])
        .strip()
        .split("\n")
    )

GHCR default selection from `docker.py:26`:

MZ_GHCR_DEFAULT = "1" if ui.env_is_truthy("CI") else "0"

Docker Hub rate limit avoidance from `mzbuild.py:274-296`:

dockerhub_username = os.getenv("DOCKERHUB_USERNAME")
dockerhub_token = os.getenv("DOCKERHUB_ACCESS_TOKEN")
# ... uses Docker Hub API or falls back to docker manifest inspect

Common Errors

Error Message	Cause	Solution
`docker manifest inspect` rate limited (429)	Docker Hub rate limits exceeded	Set `DOCKERHUB_USERNAME` and `DOCKERHUB_ACCESS_TOKEN` for authenticated access
`no such manifest`	Image not yet pushed to registry	Build the image locally with `bin/mzimage acquire <image>`
Docker daemon not running	Docker Engine not started	Start Docker with `systemctl start docker` or Docker Desktop

Compatibility Notes

Docker Hub vs GHCR: In CI, images default to GHCR (`ghcr.io/materializeinc/`). Locally, Docker Hub (`materialize/`) is the default unless `MZ_GHCR=1` is set.
Docker CLI Experimental: The `DOCKER_CLI_EXPERIMENTAL=enabled` flag is set automatically when falling back to `docker manifest inspect`.
macOS: Docker Desktop is required. Cross-compilation uses Homebrew-installed toolchains instead of ci-builder for performance.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment