Implementation:Microsoft LoRA NLU Setup Py

Template:Implementation meta

Overview

The setup.py file is the package configuration and release script for the HuggingFace Transformers library (version 4.4.2) used within the NLU example directory.

Description

This file defines the complete packaging metadata for the transformers Python package. It performs the following tasks:

• Declares all dependencies with version constraints in the _deps list and constructs a lookup table (deps dict) mapping package names to their versioned requirement strings.
• Removes stale transformers.egg-info directories to work around pip issue #5466.
• Defines a custom distutils command DepsTableUpdateCommand that auto-generates src/transformers/dependency_versions_table.py from the _deps list.
• Organizes optional dependencies into named extras groups: tf, tf-cpu, torch, flax, ja (Japanese tokenization), sklearn, retrieval, tokenizers, onnxruntime, onnx, modelcreation, serving, speech, sentencepiece, testing, docs, quality, all, dev, and torchhub.
• Specifies core install requirements: filelock, numpy, packaging, regex, requests, sacremoses, tokenizers, and tqdm.
• Registers the console script entry point transformers-cli.

Usage

Use this file when:

• Installing the transformers package from source (pip install -e .).
• Building distribution packages for PyPI (python setup.py sdist bdist_wheel).
• Updating the dependency versions table (python setup.py deps_table_update).
• Performing the release checklist described in the module docstring (steps 1 through 9).

Code Reference

Source Location

examples/NLU/setup.py (309 lines)

Key Components

Component	Type	Description
_deps	list	Master list of all dependency strings with version constraints
deps	dict	Lookup table mapping package name to full requirement string
deps_list(*pkgs)	function	Returns a list of requirement strings for the given package names
DepsTableUpdateCommand	class (Command)	Custom distutils command to regenerate dependency_versions_table.py
extras	dict	Maps extras group names to their dependency lists
install_requires	list	Core dependencies required for basic installation

Signature

# Custom command
class DepsTableUpdateCommand(Command):
    def run(self): ...

# Helper
def deps_list(*pkgs) -> list: ...

# setup() call
setup(
    name="transformers",
    version="4.4.2",
    ...
)

Import / CLI Usage

# Install from source
pip install -e .

# Install with extras
pip install -e ".[torch,testing]"

# Build distributions
python setup.py sdist bdist_wheel

# Update dependency versions table
python setup.py deps_table_update

I/O Contract

Inputs

Input	Type	Description
Command-line arguments	CLI	Standard setuptools/pip commands (install, sdist, bdist_wheel, deps_table_update)
README.md	File	Read for long_description field
src/transformers/	Directory	Package source directory discovered by find_packages("src")

Outputs

Output	Type	Description
Installed package	Package	The transformers package installed into the Python environment
dist/	Directory	Built wheel (.whl) and source (.tar.gz) distributions
src/transformers/dependency_versions_table.py	File	Auto-generated dependency lookup table (when using deps_table_update)
transformers-cli	Console script	CLI entry point for Transformers commands

Usage Examples

# Install the transformers package in editable mode with PyTorch support
pip install -e ".[torch]"

# Install with all development dependencies
pip install -e ".[dev]"

# Build the dependency versions table
python setup.py deps_table_update

# Verify the installed version
python -c "import transformers; print(transformers.__version__)"
# Output: 4.4.2

# Access dependency info programmatically
python -c "from transformers.dependency_versions_table import deps; print(deps['tokenizers'])"

Related Pages

• Environment:Microsoft_LoRA_NLU_Conda_Environment