Implementation:ArroyoSystems Arroyo Pipeline Outputs

Knowledge Sources	ArroyoSystems_Arroyo
Domains	WebUI, React, Pipelines, Streaming
Last Updated	2026-02-08 08:00 GMT

Overview

A real-time output viewer that streams pipeline results via Server-Sent Events (SSE) into an AG Grid data table with on-demand subscription, auto-column detection, and row count tracking.

Description

PipelineOutputs connects to the pipeline output SSE endpoint (/api/v1/pipelines/{id}/jobs/{jobId}/output) and displays streaming records in an AG Grid table. Key features:

SSE streaming -- Opens an EventSource connection to receive output batches. Each batch message contains a JSON OutputData with start_id, batch (JSON string of rows), and timestamps array.
Auto-schema detection -- Dynamically generates column definitions from the first batch's keys, with a fixed ID column (purple) and timestamp column (green) prepended.
Row management -- Inserts new rows at the top via applyTransaction. Caps at 10,000 rows by removing oldest rows when the limit is exceeded.
On-demand mode -- When onDemand=true, shows Start/Stop tailing buttons. The grid is hidden until data arrives.
Clear button -- Removes all rows from the grid.
Row counter -- Displays total rows read using a ref-based counter for performance (avoids React re-renders).
Object serialization -- Stringifies nested object values to JSON for display.

Usage

Used in the Results tab of both CreatePipeline (on-demand mode for preview) and PipelineDetails (on-demand mode for running pipelines with web sinks).

Code Reference

Source Location

Repository: ArroyoSystems_Arroyo
File: webui/src/routes/pipelines/PipelineOutputs.tsx

Signature

export function PipelineOutputs(props: {
  pipelineId: string;
  job: Job;
  onDemand: boolean;
}): JSX.Element;

Import

import { PipelineOutputs } from './PipelineOutputs';

I/O Contract

Inputs

Name	Type	Required	Description
pipelineId	string	Yes	Pipeline identifier
job	Job	Yes	Job object providing job.id and job.state
onDemand	boolean	Yes	Whether to show start/stop tailing controls (true) or auto-subscribe (false)

Outputs

Name	Type	Description
Data grid	React element	AG Grid table with streaming pipeline output rows

Usage Examples

<PipelineOutputs pipelineId={pipeline.id} job={job} onDemand={true} />

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment