Implementation:Microsoft LoRA NLU Environment Setup Script

Overview

NLU Environment Setup Script is an External Tool Doc that documents the two-step process for setting up the NLU experiment environment from the microsoft/LoRA repository. It uses conda to create a reproducible Python environment and then installs a modified HuggingFace Transformers v4.4.2 fork in editable mode.

Source Files

File	Lines	Description
`examples/NLU/environment.yml`	1-107	Conda environment specification
`examples/NLU/setup.py`	1-309	Modified Transformers package setup

Signature

conda env create -f environment.yml
pip install -e .

Input / Output

Direction	Description
Input	Clean conda installation; internet access for downloading packages
Output	Fully configured conda environment named `NLU` with modified Transformers and LoRA support

Step 1: Conda Environment Creation

The environment.yml file defines the NLU environment with all pinned dependencies:

cd examples/NLU
conda env create -f environment.yml
conda activate NLU

Key Conda Dependencies

python=3.7.10 -- specific Python version for reproducibility
pytorch=1.9.0 with py3.7_cuda11.1_cudnn8.0.5_0 build
torchaudio=0.9.0 and torchvision=0.10.0
numpy=1.20.2 via MKL backend

Key Pip Dependencies

loralib==0.1.1 -- LoRA layer implementations (lora.Linear, lora.Embedding)
datasets==1.9.0 -- HuggingFace Datasets for GLUE loading
tokenizers==0.10.3 -- fast tokenizer backend
deepspeed==0.5.0 -- distributed training support
accelerate==0.3.0 -- HuggingFace training utilities
scikit-learn==0.24.2 -- evaluation metrics
tensorboardx==1.8 -- training visualization
sentencepiece==0.1.96 -- subword tokenization

Step 2: Editable Install of Modified Transformers

After activating the environment, install the modified Transformers fork:

cd examples/NLU
pip install -e .

This runs setup.py which:

Identifies itself as transformers==4.4.2
Locates packages under src/ using find_packages("src")
Installs core dependencies (filelock, numpy>=1.17, regex, requests, sacremoses, tokenizers>=0.10.1,<0.11, tqdm>=4.27)
Removes any stale transformers.egg-info directory to avoid pip conflicts
Registers the transformers-cli console script entry point

Editable Mode Behavior

The -e (editable) flag creates a .egg-link file that points Python imports to the local source tree. This means:

import transformers resolves to examples/NLU/src/transformers/
Modifications to modeling_roberta.py or modeling_deberta_v2.py take effect without reinstallation
The LoRA-patched attention layers are used whenever a RoBERTa or DeBERTa V2 model is loaded with apply_lora=True

Verification

After installation, verify that the LoRA-modified Transformers is active:

import transformers
print(transformers.__version__)  # Should print 4.4.2
print(transformers.__file__)     # Should point to examples/NLU/src/transformers/__init__.py

Environment Configuration

The environment.yml also specifies a target prefix:

prefix: /opt/conda/envs/transformers

This prefix may need to be adjusted or removed for different system configurations. The name: NLU field at the top of the file determines the environment name used by conda activate NLU.

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment