Environment:Sgl project Sglang Distributed
Sgl_project_Sglang_Distributed is the distributed multi-node runtime environment for SGLang, enabling tensor parallelism and data parallelism across multiple GPUs and nodes via NCCL.
Requirements
- Multiple GPU nodes with high-speed interconnect (NVLink, InfiniBand, or RoCE)
- NCCL 2.x for GPU collective communication
- PyTorch distributed (`torch.distributed`)
- SSH access between nodes (for multi-node launches)
- Shared filesystem or model cache accessible from all nodes
- Python 3.10+
- `--tp` (tensor parallelism) and/or `--dp` (data parallelism) server arguments
Required By
See Also
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment