Implementation:ArroyoSystems Arroyo Smoke Tests
| Knowledge Sources | |
|---|---|
| Domains | Testing, SQL, Rust |
| Last Updated | 2026-02-08 08:00 GMT |
Overview
Integration smoke test harness that compiles and executes SQL test queries against the Arroyo streaming engine, validates outputs against golden files, and tests checkpoint/restore correctness.
Description
This Rust test module uses rstest with the #[files] attribute to discover all .sql files under src/test/queries/ and run each as an independent smoke test. Each test:
- Reads the SQL file and checks for special directives (
--fail,--fail=message,--pk=col1,col2). - Compiles the SQL into a LogicalProgram via
parse_and_get_arrow_program. - Runs completely -- Executes the pipeline to completion and compares output against golden output files using line-by-line JSON comparison (or Debezium merge-based comparison for CDC data).
- Runs with checkpointing -- Executes with parallelism=2, performing 3 checkpoints with compaction between epochs 2 and 3, then gracefully stops.
- Restores from checkpoint -- Resumes with parallelism=3 from checkpoint epoch 3 and runs to completion, verifying outputs match golden files again.
Key helper functions include checkpoint() for triggering and completing checkpoint rounds, compact() for Parquet compaction, merge_debezium() for applying CDC operations to build final state, and set_internal_parallelism() for adjusting operator parallelism while respecting watermark node constraints.
Input data is read from inputs/ and expected output from golden_outputs/ via path substitution variables $input_dir and $output_path in the SQL files.
Usage
Run via cargo test within the arroyo-sql-testing crate. Tests are data-driven: adding a new .sql file with corresponding input/golden output files creates a new test case automatically.
Code Reference
Source Location
- Repository: ArroyoSystems_Arroyo
- File: crates/arroyo-sql-testing/src/smoke_tests.rs