Principle:MaterializeInc Materialize Console Release Coordination
| Knowledge Sources | misc/python/materialize/release/cut_release.py |
|---|---|
| Domains | Release Engineering, Multi-Repository Coordination, Container Registries |
| Last Updated | 2026-02-08 |
Overview
Multi-repository release synchronization coordinates releases across dependent repositories by tagging and polling for build completion before proceeding.
Description
Console Release Coordination addresses the challenge of releasing software that spans multiple git repositories with build dependencies. In Materialize's architecture, the main materialize repository depends on the console repository: the Materialize Docker image embeds a specific version of the console UI. This creates a cross-repository release ordering constraint: the console must be built and published to DockerHub before the main Materialize release can reference it.
The coordination pattern involves three phases:
- Tag propagation: The release script clones the console repository, creates an annotated git tag matching the target version, and pushes it. This triggers the console's own CI/CD pipeline to build and publish the console Docker image.
- Artifact polling: After pushing the tag, the script enters a polling loop, repeatedly checking DockerHub for the console image using
docker manifest inspect. This accounts for the asynchronous nature of CI builds -- the console image typically takes ~15 minutes to appear. - Reference pinning: Once the console image is confirmed available, the main repository updates its Dockerfile and Helm chart templates to reference the exact console image version.
This pattern handles the fallback authentication problem: the console repo clone first attempts HTTPS (with GIT_TERMINAL_PROMPT=0 to prevent interactive auth prompts), and falls back to SSH if HTTPS fails. This supports both CI environments (which may use tokens) and developer workstations (which may use SSH keys).
Usage
Apply this principle when your release process involves multiple repositories where one repository's artifacts must be published before another can reference them. This is common in microservice architectures, monorepo-adjacent setups where UI and backend are in separate repos, and any system where Docker images are composed from multiple independently-built components.
Theoretical Basis
This pattern is a concrete instance of distributed workflow orchestration with polling-based synchronization. The theoretical foundation includes:
Dependency graph execution: The release forms a directed acyclic graph (DAG) where the console build is a prerequisite for the main build. The simplest correct execution strategy is topological-order execution with blocking waits.
Polling vs. event-driven synchronization: The implementation uses polling (checking DockerHub every 60 seconds) rather than webhooks or event subscriptions. While less efficient, polling is:
- Simpler -- no webhook infrastructure needed.
- More robust -- no missed events, no callback URL management.
- Self-healing -- transient failures in the poll are automatically retried.
Idempotent tag creation: The script checks for an existing tag before creating one (git tag -l version), making the console tagging step idempotent. This is critical for release retry scenarios where the console tag may already exist from a previous failed attempt.
The polling loop implements a busy-wait with backoff pattern (fixed 60-second interval), which is acceptable given the expected ~15 minute build time and the low frequency of release operations.