Implementation:TobikoData Sqlmesh CICD Update Pr Environment
| Knowledge Sources | |
|---|---|
| Domains | Data_Engineering, CICD |
| Last Updated | 2026-02-07 00:00 GMT |
Overview
Concrete implementation for creating and updating isolated virtual data environments for pull requests provided by SQLMesh.
Description
The update_pr_environment command and controller method automate the creation of PR-specific virtual data environments within GitHub Actions workflows. It generates a sanitized environment name from the PR number, creates a plan comparing PR changes to production, applies the plan to create the environment with appropriate data loading strategies, updates a GitHub Check Run with environment creation status, and posts a PR comment with the environment name and optional deployment instructions. The implementation enables isolated testing of data transformation changes before production deployment.
Usage
Use update_pr_environment as a GitHub Actions workflow step after tests pass to create an isolated data environment where PR changes can be reviewed and validated. This typically runs after successful test execution and before generating the production plan.
Code Reference
Source Location
- Repository: sqlmesh
- File: sqlmesh/integrations/github/cicd/command.py:L143-148 (CLI), sqlmesh/integrations/github/cicd/controller.py:L742-762 (controller method)
Signature
# CLI Command
@github.command()
@click.pass_context
@cli_analytics
def update_pr_environment(ctx: click.Context) -> None:
"""Creates or updates the PR environments"""
if not _update_pr_environment(ctx.obj["github"]):
raise CICDBotError(
"Failed to update PR environment. See Pull Requests Checks for more information."
)
# Controller Method
def update_pr_environment(self) -> None:
"""
Creates a PR environment from the logic present in the PR. If the PR contains changes that are
uncategorized, then an error will be raised.
"""
self._console.consume_captured_output() # clear output buffer
self._context.apply(self.pr_plan) # will raise if PR environment creation fails
# update PR info comment
vde_title = "- :eyes: To **review** this PR's changes, use virtual data environment:"
comment_value = f"{vde_title}\n - `{self.pr_environment_name}`"
if self.bot_config.enable_deploy_command:
full_command = f"{self.bot_config.command_namespace or ''}/deploy"
comment_value += f"\n- :arrow_forward: To **apply** this PR's plan to prod, comment:\n - `{full_command}`"
dedup_regex = vde_title.replace("*", r"\*") + r".*"
updated_comment, _ = self.update_sqlmesh_comment_info(
value=comment_value,
dedup_regex=dedup_regex,
)
if updated_comment:
self._append_output("created_pr_environment", "true")
Import
from sqlmesh.integrations.github.cicd.command import update_pr_environment
from sqlmesh.integrations.github.cicd.controller import GithubController
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| ctx | click.Context | Yes | Click context containing initialized GithubController in ctx.obj["github"] |
Outputs
| Name | Type | Description |
|---|---|---|
| None | None | Command creates PR environment, updates GitHub Check Run, and posts PR comment; raises CICDBotError if creation fails |
Usage Examples
Basic Usage
# In GitHub Actions workflow:
# - name: Create PR Environment
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# run: sqlmesh_cicd bot github update-pr-environment
# Programmatic usage:
from sqlmesh.integrations.github.cicd.controller import GithubController, GithubCheckStatus
controller = GithubController(
paths=["path/to/project"],
token="github_token"
)
# Update check to in-progress
controller.update_pr_environment_check(status=GithubCheckStatus.IN_PROGRESS)
try:
# Create or update PR environment
controller.update_pr_environment()
# Update check to completed with success
controller.update_pr_environment_check(status=GithubCheckStatus.COMPLETED)
except Exception as e:
# Update check to completed with failure
controller.update_pr_environment_check(
status=GithubCheckStatus.COMPLETED,
exception=e
)
# Example PR comment posted:
# 🤖 **SQLMesh Bot Info** 🤖
# - 👀 To **review** this PR's changes, use virtual data environment:
# - `my_repo_123`
# - ▶️ To **apply** this PR's plan to prod, comment:
# - `/deploy`
# Example GitHub Check Run summary:
# Here is a summary of data that has been loaded into the PR environment `my_repo_123`
# and could be deployed to `prod`.
#
# ### Added
# - `db.customers_v2` (Non-breaking)
# **Kind:** INCREMENTAL_BY_TIME_RANGE
# **Dates loaded in PR:** [2024-01-01 - 2024-01-31]
#
# ### Directly Modified
# - `db.orders` (Breaking)
# **Kind:** FULL
# **Dates loaded in PR:** [2024-01-01 - 2024-01-31]
# **Dates *not* loaded in PR:** [2024-02-01 - 2024-12-31]