Implementation:MaterializeInc Materialize Composition Run

Knowledge Sources	Materialize mzcompose framework, Docker Compose API, Python subprocess module
Domains	Test Execution, Container Orchestration, Docker Compose, Python
Last Updated	2026-02-08

Overview

Concrete Python method for container-orchestrated test command execution provided by Materialize's Composition.run(), which delegates to docker compose run with support for detached mode, environment injection, output capture, entrypoint override, and exit code propagation.

Description

The run() method on the Composition class executes a one-off command inside a Docker Compose service container. It is the primary mechanism by which Materialize's CI test framework runs test commands in isolated, ephemeral environments.

The method wraps docker compose run and adds several features:

1. Automatic Image Rebuilding: Unlike raw docker compose run, this method rebuilds any services whose definitions have changed (similar to docker compose up) before executing the command. This is handled by the underlying invoke() method.
2. Entrypoint Override: The entrypoint parameter allows overriding the container's default entrypoint, enabling arbitrary commands to be run in any service container.
3. Environment Injection: The env_extra parameter passes additional environment variables to the container. These are merged with the current process environment using os.environ | env_extra and passed via -e flags.
4. Output Capture Modes: Three capture modes are supported:
   • capture=True: Captures stdout only.
   • capture_stderr=True: Captures stderr only.
   • capture_and_print=True: Captures combined stdout+stderr while also printing to the console in real time.
5. Detached Mode: The detach parameter runs the container in the background.
6. Container Cleanup: The rm parameter removes the container after the run completes.
7. Exit Code Checking: The check parameter (default True) raises an exception if the command exits with a non-zero status.
8. Silent Mode: The silent parameter suppresses command output for commands whose output is not interesting.

Usage

This method is used throughout Materialize's test workflows to execute test drivers, database commands, and utility scripts inside composition service containers. It is called from mzcompose workflow functions defined in mzcompose.py files within test directories.

Code Reference

Source Location

misc/python/materialize/mzcompose/composition.py, lines 844-893.

Signature

def run(
    self,
    service: str,
    *args: str,
    detach: bool = False,
    rm: bool = False,
    env_extra: dict[str, str] = {},
    capture: bool = False,
    capture_stderr: bool = False,
    capture_and_print: bool = False,
    stdin: str | None = None,
    entrypoint: str | None = None,
    check: bool = True,
    silent: bool = False,
) -> subprocess.CompletedProcess:
    """Run a one-off command in a service.

    Delegates to `docker compose run`. See that command's help for details.
    Note that unlike `docker compose run`, any services whose definitions
    have changed are rebuilt (like `docker compose up` would do) before the
    command is executed.

    Args:
        service: The name of a service in the composition.
        args: Arguments to pass to the service's entrypoint.
        detach: Run the container in the background.
        stdin: read STDIN from a string.
        env_extra: Additional environment variables to set in the container.
        rm: Remove container after run.
        capture: Capture the stdout of the `docker compose` invocation.
        capture_stderr: Capture the stderr of the `docker compose` invocation.
        capture_and_print: Print during execution and capture the
            stdout+stderr of the `docker compose` invocation.
    """

Import

from materialize.mzcompose.composition import Composition

I/O Contract

Inputs

Name	Type	Default	Description
service	str	(required)	Name of a service defined in the Docker Compose composition.
*args	str (variadic)	(none)	Arguments passed to the service's entrypoint or overridden command.
detach	bool	False	Run the container in the background (adds --detach flag).
rm	bool	False	Remove the container after the run completes (adds --rm flag).
env_extra	dict[str, str]	{}	Additional environment variables to inject into the container via -e flags.
capture	bool	False	Capture stdout of the docker compose invocation.
capture_stderr	bool	False	Capture stderr of the docker compose invocation.
capture_and_print	bool	False	Capture stdout+stderr while also printing to console.
stdin	str or None	None	String to pass as stdin to the container process.
entrypoint	str or None	None	Override the container's default entrypoint (adds --entrypoint flag).
check	bool	True	If True, raises subprocess.CalledProcessError on non-zero exit.
silent	bool	False	Suppress command output.

Outputs

Name	Type	Description
Return value	subprocess.CompletedProcess	Contains returncode, stdout (if captured), and stderr (if captured).

Usage Examples

Running a test driver in a composition:

from materialize.mzcompose.composition import Composition

# Inside a workflow function:
def workflow_default(c: Composition):
    c.up("materialized", "testdrive")
    c.run("testdrive", "test_file.td")

Running with environment variables and capturing output:

result = c.run(
    "testdrive",
    "regression_test.td",
    env_extra={"MZ_SOFT_ASSERTIONS": "1"},
    capture=True,
)
print(f"Test output: {result.stdout}")

Running with a custom entrypoint:

result = c.run(
    "materialized",
    "-c", "SELECT 1;",
    entrypoint="psql",
    capture=True,
)

Running in detached mode with container cleanup:

c.run("load-generator", detach=True, rm=True)

Running with stdin input:

result = c.run(
    "materialized",
    entrypoint="psql",
    stdin="SELECT version();\n",
    capture=True,
)

Related Pages

Implements Principle

• Principle:MaterializeInc_Materialize_Composition_Execution

Requires Environment

• Environment:MaterializeInc_Materialize_Docker_Compose_Runtime