Implementation:Allenai Open instruct Beaker Experiment Workflow
| Knowledge Sources | |
|---|---|
| Domains | CI_CD, Infrastructure |
| Last Updated | 2026-02-07 02:00 GMT |
Overview
GitHub Actions workflow that builds Docker images, launches integration test experiments on Beaker, and promotes verified images upon success.
Description
The beaker-experiment.yml workflow is the primary CI/CD pipeline for the open-instruct project. Triggered on merge queue events or manual dispatch, it detects which source files changed (DPO, finetune, GRPO paths), builds a Docker image, launches corresponding integration-test experiments on Beaker with GPU resources, waits for completion with a 20-minute timeout, and on success pushes the verified image as open_instruct_auto. On failure, it posts a summary and notifies Slack.
Usage
This workflow runs automatically when code enters the merge queue. It can also be triggered manually via GitHub's workflow_dispatch. It is the integration gate that ensures all training code changes pass real GPU experiments before being merged.
Code Reference
Source Location
- Repository: Allenai_Open_instruct
- File: .github/workflows/beaker-experiment.yml
- Lines: 1-382
Signature
name: Beaker Experiment Launch
on:
merge_group:
workflow_dispatch:
jobs:
launch-experiment:
name: Launch Beaker Experiment
runs-on: 8-Core-XL-Runner-Ubuntu-Latest
timeout-minutes: 35
steps:
- name: Checkout repository
- name: Detect file changes
- name: Setup Python 3.12 and uv
- name: Build Docker image
- name: Launch experiments (GRPO, DPO, SFT)
- name: Wait for experiments (20-min timeout)
- name: Push verified image
- name: Notify on failure
Import
# N/A - GitHub Actions workflow, triggered by merge_group or workflow_dispatch events
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| merge_group event | GitHub Event | No | Triggered automatically when PRs enter merge queue |
| workflow_dispatch | GitHub Event | No | Manual trigger via GitHub UI |
| BEAKER_TOKEN | Secret | Yes | Authentication for Beaker cluster |
| DOCKER credentials | Secret | Yes | Docker Hub push credentials |
Outputs
| Name | Type | Description |
|---|---|---|
| Docker image | Beaker Image | Verified image pushed as open_instruct_auto |
| Experiment results | Beaker Experiments | GRPO, DPO, and SFT integration test results |
| Slack notification | Message | Failure notification to team channel |
Usage Examples
Automatic Trigger
# This workflow runs automatically when a PR enters the merge queue.
# No manual action needed - GitHub triggers it on merge_group events.
# To trigger manually via GitHub CLI:
gh workflow run beaker-experiment.yml
Change Detection Logic
# The workflow detects which experiments to run based on changed files:
# - DPO changes: open_instruct/dpo*.py, open_instruct/olmo_core*.py
# - SFT changes: open_instruct/finetune.py
# - GRPO changes: always runs (primary integration target)