Principle:Apache Druid SQL Ingestion Monitoring

Knowledge Sources	Apache Druid Druid SQL Statements API
Domains	Data_Ingestion, Task_Management, Monitoring
Last Updated	2026-02-10 00:00 GMT

Overview

A task monitoring principle that tracks the execution progress of asynchronous SQL ingestion tasks through polling and stage visualization.

Description

SQL Ingestion Monitoring provides real-time visibility into MSQ task execution. After a task is submitted, the monitoring system polls the Druid API to track:

Task state: RUNNING, SUCCESS, FAILED
Stage progress: Number of stages, rows processed per stage, worker allocation
Timing: Duration of each stage, total elapsed time
Errors: Error messages and affected stages on failure

The monitoring uses two endpoints:

/druid/indexer/v1/task/{id}/reports — detailed stage reports
/druid/v2/sql/statements/{id}?detail=true — SQL-level status with result metadata

Usage

Use this principle immediately after SQL task submission. Monitoring continues automatically until the task reaches a terminal state (SUCCESS or FAILED).

Theoretical Basis

Task monitoring follows a state machine polling pattern:

Poll loop:
  GET /druid/v2/sql/statements/{id}?detail=true → { state, stages, duration, error }

State machine:
  ACCEPTED → RUNNING → SUCCESS
                     → FAILED

Stage tracking:
  Each stage has: inputRows, outputRows, duration, workerCount
  Stages execute as a DAG (directed acyclic graph)

Related Pages

Implemented By

Implementation:Apache_Druid_IngestionProgressDialog

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment