Principle:Treeverse LakeFS Hook Guarded Merge
| Knowledge Sources | |
|---|---|
| Domains | Data_Quality, Automation |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
Merge operations with automated pre-merge validation gates enable controlled data promotion workflows where data must pass quality checks before entering production branches.
Description
Pre-merge hooks create a promotion gateway between branches. When a merge is initiated (e.g., from a feature branch to main), lakeFS evaluates all pre-merge hooks defined on the destination branch before allowing the merge to proceed. This is analogous to CI/CD pipelines for code, where automated tests must pass before a pull request can be merged.
The hook-guarded merge workflow typically follows this pattern:
- Data is written and committed on an isolated feature or staging branch
- Pre-commit hooks on the feature branch validate individual commits
- When the data is ready for production, a merge to
mainis initiated - Pre-merge hooks on
mainexecute, performing cross-branch validation - If all hooks pass, the merge completes; if any fail, the merge is rejected (HTTP 412)
- Post-merge hooks fire asynchronously to trigger downstream pipelines and notifications
Pre-merge hooks can perform validations that are different from and complementary to pre-commit hooks:
- Cross-branch consistency -- Verify that the incoming data is compatible with existing data on the target branch
- Schema compatibility -- Check that schema changes are backward-compatible with downstream consumers
- Data freshness -- Ensure the data being promoted is not stale relative to the production timeline
- Aggregate quality checks -- Validate metrics across the entire dataset, not just the changed files
- Approval verification -- Check that required human approvals have been recorded (via metadata or external systems)
Usage
Use hook-guarded merges when you need to:
- Implement promotion pipelines -- Gate data promotion from staging to production on automated quality checks
- Enforce cross-branch compatibility -- Validate that merged data is consistent with the destination branch
- Require multi-stage validation -- Apply different validation rules at commit time versus merge time
- Create release workflows -- Control when and how data enters release branches
- Trigger production pipelines -- Use post-merge hooks to kick off ETL, analytics, or reporting pipelines
Theoretical Basis
The hook-guarded merge model implements a controlled promotion pattern:
Two-phase validation: Pre-commit hooks validate data at the point of creation; pre-merge hooks validate data at the point of promotion. This two-phase approach catches errors at the earliest possible stage while also enforcing holistic quality standards at the integration point.
Branch as environment: In this model, branches represent deployment environments (development, staging, production). Merging data between branches is equivalent to promoting code between environments. Pre-merge hooks serve the same role as deployment gates in CI/CD pipelines.
Destination-defined policy: Pre-merge hooks are evaluated from the destination branch, not the source. This means the production branch defines its own acceptance criteria. Data producers do not need to know what validation will be performed -- they simply initiate the merge, and the destination branch's hooks determine acceptance.
Immutable audit trail: Whether a merge succeeds or fails, the hook execution results are recorded and can be queried via the Actions API. This creates an auditable record of every promotion attempt, including those that were rejected.