Implementation:MaterializeInc Materialize CLI Cloudbench
| Knowledge Sources | |
|---|---|
| Domains | CLI, Benchmarking, Cloud Infrastructure |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
The CLI Cloudbench tool launches Materialize performance benchmarks on cloud infrastructure (AWS EC2) using scratch instances, supporting multi-revision comparison.
Description
This module provides the bin/cloudbench CLI for running benchmarks against specific Git revisions on AWS EC2 scratch instances. It supports two subcommands: start (launch benchmark instances with configurable profiles, trials, and revisions) and check (monitor running benchmarks and collect results). The tool uses the materialize.scratch module to provision EC2 instances, supports both "basic" (Materialize-only) and "confluent" (with Kafka/Schema Registry) machine profiles, and can optionally upload CSV results with metadata to S3.
Usage
Use this tool when performing cloud-based performance benchmarking of Materialize across different Git revisions. It requires AWS credentials and the scratch infrastructure to be configured.
Code Reference
Source Location
- Repository: MaterializeInc_Materialize
- File: misc/python/materialize/cli/cloudbench.py
Signature
def main() -> None: ...
def configure_start(parser: argparse.ArgumentParser) -> None: ...
def start(args: argparse.Namespace) -> None: ...
def configure_check(parser: argparse.ArgumentParser) -> None: ...
def check(args: argparse.Namespace) -> None: ...
Import
from materialize.cli.cloudbench import main
I/O Contract
| Input | Type | Description |
|---|---|---|
| --profile | str |
Machine profile: "basic" (Materialize only) or "confluent" (with Kafka stack) |
| --trials | int |
Number of benchmark trials per revision (default: 1) |
| --revs | str |
Comma-separated Git revisions to benchmark (default: "HEAD") |
| bench_script | str |
Path to the benchmark script with optional arguments |
| --append_metadata | bool |
Whether to append metadata to CSV rows before S3 upload |
| Output | Type | Description |
|---|---|---|
| EC2 instances | AWS resources | Scratch instances running benchmarks |
| CSV results | file | Benchmark results optionally uploaded to S3 |
Usage Examples
# Start a benchmark comparing two revisions
bin/cloudbench start --profile confluent --trials 3 --revs "HEAD,HEAD~5" bench_script.sh
# Check benchmark status
bin/cloudbench check