Implementation:Sgl project Sglang Sgl Runtime Init

Knowledge Sources	SGLang
Domains	Frontend_DSL, LLM_Serving, Configuration
Last Updated	2026-02-10 00:00 GMT

Overview

Concrete tool for initializing SGLang frontend DSL backends (local Runtime or remote RuntimeEndpoint) for generation program execution.

Description

sgl.Runtime creates a local backend that spawns a server process with the specified model. sgl.RuntimeEndpoint creates a remote backend pointing to an existing server URL. After creation, sgl.set_default_backend registers the backend globally so @sgl.function programs can execute.

Usage

Call sgl.Runtime for local execution or sgl.RuntimeEndpoint for remote server access. Always call sgl.set_default_backend to register the backend before running any SGLang programs.

Code Reference

Source Location

• Repository: sglang
• File: python/sglang/lang/backend/runtime_endpoint.py
• Lines: L356-545 (Runtime), L26-355 (RuntimeEndpoint)
• set_default_backend: python/sglang/lang/api.py:L49-50

Signature

# Local Runtime (spawns server process)
sgl.Runtime(
    model_path: str,
    log_level: str = "error",
    launch_timeout: float = 300.0,
    **kwargs,  # Additional ServerArgs parameters
) -> Runtime

# Remote RuntimeEndpoint (connects to existing server)
sgl.RuntimeEndpoint(
    base_url: str,
    api_key: Optional[str] = None,
) -> RuntimeEndpoint

# Register as default
sgl.set_default_backend(backend: BaseBackend)

Import

import sglang as sgl

# Local:
runtime = sgl.Runtime(model_path="...")
sgl.set_default_backend(runtime)

# Remote:
endpoint = sgl.RuntimeEndpoint("http://localhost:30000")
sgl.set_default_backend(endpoint)

I/O Contract

Inputs

Name	Type	Required	Description
model_path	str	Yes (Runtime)	Model to load locally
base_url	str	Yes (RuntimeEndpoint)	URL of running SGLang server
api_key	Optional[str]	No	API key for authentication
log_level	str	No	Logging level (default: "error")

Outputs

Name	Type	Description
backend	Runtime or RuntimeEndpoint	Backend instance ready for set_default_backend

Usage Examples

Local Runtime

import sglang as sgl

# Start local model server
runtime = sgl.Runtime(model_path="meta-llama/Llama-3.1-8B-Instruct")
sgl.set_default_backend(runtime)

# Now @sgl.function programs can execute
@sgl.function
def chat(s, question):
    s += sgl.user(question)
    s += sgl.assistant(sgl.gen("answer", max_tokens=128))

state = chat.run(question="What is AI?")
print(state["answer"])

# Cleanup
runtime.shutdown()

Remote RuntimeEndpoint

import sglang as sgl

# Connect to existing server
endpoint = sgl.RuntimeEndpoint("http://localhost:30000")
sgl.set_default_backend(endpoint)

# Same programs work with remote backend
state = chat.run(question="What is AI?")

Related Pages

Implements Principle

• Principle:Sgl_project_Sglang_Frontend_Backend_Initialization

Requires Environment

• Environment:Sgl_project_Sglang_Python_Dependencies