Principle:Sgl project Sglang Frontend Backend Initialization
| Knowledge Sources | |
|---|---|
| Domains | Frontend_DSL, LLM_Serving, Configuration |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
A backend initialization pattern that connects the SGLang frontend DSL to either a local model runtime or a remote server endpoint for program execution.
Description
The SGLang frontend DSL (@sgl.function, sgl.gen, sgl.fork) requires a backend to execute generation programs. Two backend types are available: Runtime (spawns a local server process with the model loaded) and RuntimeEndpoint (connects to an existing remote SGLang server). After creating a backend, sgl.set_default_backend makes it available to all @sgl.function programs. This decouples program definition from execution environment.
Usage
Initialize a backend at the start of any SGLang frontend DSL program. Use Runtime for self-contained scripts and RuntimeEndpoint when connecting to a pre-deployed server.
Theoretical Basis
The pattern follows a Strategy design: the frontend DSL defines computation graphs (programs), and the backend provides the execution strategy. This separation enables:
- Same program code runs locally or remotely
- Backend can be swapped without changing program logic
- Multiple backends can be used in the same application
Backend types:
- Runtime — Local: spawns a server process, loads model, auto-shutdown on garbage collection
- RuntimeEndpoint — Remote: connects to an existing server via HTTP