Principle:Sail sg LongSpec Environment Setup
| Knowledge Sources | |
|---|---|
| Domains | DevOps, Environment_Management |
| Last Updated | 2026-02-14 05:00 GMT |
Overview
Principle for establishing the software environment required for GLIDE draft model training and speculative decoding inference.
Description
Environment Setup ensures all required Python packages, CUDA libraries, and system dependencies are installed and compatible. The LongSpec repository has strict version requirements for:
- PyTorch ecosystem: torch==2.6.0, triton==3.2.0
- HuggingFace stack: transformers==4.51.1, datasets==2.19.1, peft==0.13.2, accelerate==1.0.1
- Flash Attention: flash_attn==2.6.3 (requires CUDA-compatible GPU)
- DeepSpeed: Installed separately, required for distributed training
- Liger Kernel: liger_kernel==0.3.1 for fused cross-entropy loss
- Experiment tracking: wandb==0.19.11
Training and inference have separate requirements files with different dependency sets.
Usage
Must be completed before any training or inference. The environment is set up once per machine/container and reused across all experiments.
Theoretical Basis
Dependency pinning ensures reproducibility. All versions are exact-pinned (==) rather than minimum-version (>=) to prevent unexpected behavior from package updates.
Related Pages
Implemented By
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment