Principle:TobikoData Sqlmesh CICD Production Plan Generation
| Knowledge Sources | |
|---|---|
| Domains | Data_Engineering, CICD |
| Last Updated | 2026-02-07 00:00 GMT |
Overview
Automatic generation and display of production deployment plans showing exactly what changes will be applied before execution.
Description
Production plan generation creates a preview of changes that would be deployed to production if a PR is merged. By analyzing differences between the PR environment and production, generating a detailed change report, and surfacing it through GitHub Check Runs and PR comments, teams can review deployment impact before approving merges. This provides transparency and enables informed decision-making about production deployments.
This principle addresses the problem of deployment surprises where teams don't fully understand what will change in production until after deployment. Plan generation brings infrastructure-as-code review practices to data transformations.
Usage
Use production plan generation when:
- Reviewing deployment impact before merging to production branches
- Understanding which models will be rebuilt vs virtually updated
- Identifying data gaps between PR environments and production
- Estimating deployment time and resource usage
- Communicating changes to stakeholders during review
- Detecting breaking changes requiring careful coordination
- Planning backfill operations for incremental models
Theoretical Basis
Production plan generation implements the preview-before-apply pattern from infrastructure-as-code tools. The process consists of:
Plan Generation:
- Create a plan comparing PR environment to production environment
- Enable no_gaps=True to detect missing data intervals
- Identify added, removed, and modified models
- Determine categorization (breaking/non-breaking/forward-only)
- Calculate which models need data backfills
Change Analysis:
- Added models: New transformations in this PR
- Removed models: Transformations deleted in this PR
- Directly modified: Models with changed SQL logic
- Indirectly modified: Models whose upstream dependencies changed
- Metadata updated: Models with only metadata changes (tags, descriptions)
Interval Analysis: For incremental models, calculate:
- Intervals already loaded in PR environment
- Missing intervals between PR and production
- Whether deployment will be virtual (no data movement) or require backfill
Format Options:
- Breaking changes: Require full table rebuilds, may impact downstream
- Non-breaking changes: Can be deployed as views pointing to new versions
- Forward-only changes: Update in-place without versioning
Deployment Preview: Generate human-readable summary showing:
- List of models by change type and category
- Date ranges that need backfilling
- Models that will use virtual updates (no data movement)
- Models requiring full refresh
- Warnings about potential impacts
Communication Channels:
- GitHub Check Run: Real-time plan display in PR checks tab
- PR Comment: Collapsible summary with full plan details
- Console Output: Detailed plan for CI logs
The plan generation provides all information needed to make deployment decisions without executing any changes. This enables safe review workflows where approvers understand exactly what will happen in production.
For forward-only deployments, the plan also includes tips on retroactively applying changes to historical date ranges using restatement commands.