Implementation:Ucbepic Docetl WebSocket Run Pipeline

Knowledge Sources	DocETL
Domains	Web_Application, Real_Time_Processing
Last Updated	2026-02-08 01:40 GMT

Overview

Concrete WebSocket endpoint and UI components for real-time pipeline execution in the DocETL playground.

Description

The backend WebSocket /ws/run_pipeline/{namespace} endpoint receives pipeline configuration, runs DSLRunner in a background thread, and streams console output and results via WebSocket messages. The frontend Output component displays real-time console output, final results in a paginated table, and optional visualization for reduce/resolve operations.

Usage

Trigger from the playground UI's "Run All" button, or connect directly to the WebSocket endpoint with a pipeline configuration JSON message.

Code Reference

Source Location

Repository: docetl
File: server/app/routes/pipeline.py (L378-466), website/src/components/Output.tsx (L483-814)

Signature

@router.websocket("/ws/run_pipeline/{namespace}")
async def websocket_run_pipeline(websocket: WebSocket, namespace: str):
    """WebSocket endpoint for real-time pipeline execution.

    Receives: Pipeline config JSON
    Sends: {"type": "output"|"result"|"error", ...}
    """

Import

# Backend: included via app.include_router(pipeline.router)

I/O Contract

Inputs

Name	Type	Required	Description
namespace	str	Yes	URL path parameter for isolation
pipeline config	JSON	Yes	Pipeline configuration sent via WebSocket

Outputs

Name	Type	Description
output messages	{"type": "output", "data": str}	Real-time console output
result message	{"type": "result", "data": {"cost": float}}	Final execution result
error message	{"type": "error", "message": str}	Execution error

Usage Examples

// Frontend WebSocket connection
const ws = new WebSocket(`ws://localhost:8000/ws/run_pipeline/${namespace}`);
ws.onopen = () => ws.send(JSON.stringify(pipelineConfig));
ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);
  if (msg.type === "output") console.log(msg.data);
  if (msg.type === "result") console.log(`Cost: $${msg.data.cost}`);
};

Related Pages

Implements Principle

Principle:Ucbepic_Docetl_Pipeline_Execution_Streaming

Requires Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment