Principle:ClickHouse ClickHouse Remote Syslog Logging
ClickHouse_ClickHouse
ClickHouse_ClickHouse_Poco_RemoteSyslogChannel
ClickHouse_ClickHouse_Poco_RemoteSyslogListener
| Knowledge Sources | |
|---|---|
| Domains | Networking, Logging |
| Last Updated | 2026-02-08 00:00 GMT |
Purpose
Structured transmission of log messages over the network using the syslog protocol, enabling centralized log collection from distributed ClickHouse nodes. Remote syslog supports both the modern RFC 5424 format and the legacy BSD (RFC 3164) format, sending categorized log events over UDP to a remote syslog collector.
Theoretical Basis
Remote syslog logging is built on the following standards and concepts:
- RFC 5424 (The Syslog Protocol): Defines the modern syslog message format with structured data, precise timestamps (ISO 8601), hostname, application name, process ID, and message ID fields. Messages are prefixed with a priority value encoding both facility and severity.
- RFC 3164 (BSD Syslog Protocol): The legacy format using a simpler timestamp (Mmm dd HH:MM:SS), hostname, and message text. Widely supported by older syslog implementations.
- RFC 5426 (Transmission of Syslog Messages over UDP): Specifies UDP as the transport layer for syslog messages, using the well-known port 514.
- Priority encoding: The priority value is computed as
facility * 8 + severity, packed into a single integer. The lower 3 bits encode severity (0=Emergency through 7=Debug), and the upper bits encode the facility (kern, user, mail, daemon, auth, etc.). - Producer/Consumer pattern: The listener architecture uses a notification queue where a UDP receiver thread enqueues raw messages and separate parser threads dequeue and process them, decoupling I/O from parsing.
Key Properties
- Messages are transmitted over UDP, which is connectionless and fire-and-forget -- message loss is acceptable for logging workloads
- Facility codes classify the source subsystem (kern, user, daemon, auth, local0-local7, etc.)
- Severity codes range from Emergency (0) to Debug (7), mapping to Poco `Message::Priority` levels
- The channel auto-opens on first `log` call, resolving the target address lazily
- BSD format omits version, structured data, process ID, and message ID fields compared to RFC 5424
- The listener supports multi-threaded parsing via a configurable thread pool (1-15 parser threads)
- Structured data (RFC 5424) can be passed through the message properties
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment