Principle:Lm sys FastChat Environment Setup
| Field | Value |
|---|---|
| Page Type | Principle |
| Title | Environment Setup |
| Repository | lm-sys/FastChat |
| Workflow | Vicuna SFT Finetuning |
| Domains | ML Training, Python Packaging, Dependency Management |
| Knowledge Sources | pyproject.toml, pip documentation, Python packaging standards |
| Last Updated | 2026-02-07 14:00 GMT |
Overview
This principle covers the theory and best practices for setting up reproducible Python environments tailored to machine learning training workloads. In large-scale ML projects such as Vicuna SFT fine-tuning, correct environment configuration is a prerequisite for all downstream steps: data loading, model initialization, distributed training, and checkpoint saving all depend on having the right packages at compatible versions.
Description
Setting up a Python environment for ML training involves several interconnected concerns:
Virtual Environments
Isolating project dependencies from the system Python and from other projects is essential. Virtual environments (via venv, conda, or similar tools) ensure that:
- Package versions are pinned per-project and do not conflict with other workloads on the same machine.
- Reproducibility is maintained across developer machines and production training clusters.
- System-level packages are not inadvertently modified.
Dependency Management with Extras
Modern Python packaging supports optional dependency groups (also called "extras") defined in pyproject.toml. This pattern allows a single package to declare multiple sets of dependencies for different use cases:
- A base set of dependencies required for all functionality.
- A train extra that brings in training-specific libraries such as mixed-precision kernels, experiment tracking, and efficient attention implementations.
- A model_worker extra for serving and inference, including model acceleration and quantization libraries.
- A dev extra for linting and formatting tools.
This design keeps the installation lightweight for users who only need inference or serving, while providing everything needed for training when the appropriate extra is requested.
Editable Installs
During active development, editable installs (pip install -e .) allow changes to the source code to take effect immediately without reinstalling. This is particularly important for ML training workflows where:
- Researchers frequently iterate on data preprocessing and training scripts.
- The training code lives within the installed package namespace.
- Debugging requires seamless access to source files.
Version Pinning and Compatibility
ML training environments are sensitive to version mismatches. Key considerations include:
- Transformer library versions must match model architecture expectations (e.g., RoPE scaling support requires
transformers>=4.31.0). - Flash Attention requires specific CUDA toolkit and GPU architecture compatibility.
- PyTorch versions must be compatible with FSDP features used during distributed training.
- WandB or other tracking libraries must be present if experiment logging is enabled.
Usage
When preparing to run the Vicuna SFT fine-tuning workflow:
- Create and activate an isolated virtual environment.
- Install the package with the appropriate extras for your role (training, serving, or development).
- Verify that GPU-dependent packages (Flash Attention, PyTorch with CUDA) are correctly linked to the available hardware.
- Confirm version compatibility between transformers, torch, and accelerate.
Theoretical Basis
The principle of environment isolation derives from software engineering best practices for reproducible builds. In the ML context, this is reinforced by the observation that training runs are expensive (measured in GPU-hours and dollars), and environment-related failures mid-training are costly. The extras pattern in Python packaging (PEP 508) provides a declarative way to manage conditional dependencies, reducing the surface area for human error during setup.