Principle:Vespa engine Vespa Logger Handler Installation

Logger Handler Installation

Logging Observability

Overview

Logger handler installation replaces the default Java Util Logging (JUL) handlers with a custom VespaLogHandler that integrates with Vespa's logging infrastructure. The handler intercepts all log records produced by any JUL logger in the process, applies level control and reject filtering, and outputs messages in Vespa's standardized tab-delimited format.

This step is the bridge between the standard Java logging API and Vespa's specialized logging pipeline. After handler installation, all calls to java.util.logging.Logger methods (info(), warning(), etc.) flow through Vespa's custom handler.

Motivation

Java's built-in logging framework (JUL) provides a handler abstraction that allows custom output processing. However, the default ConsoleHandler has several limitations for a distributed system like Vespa:

• It outputs in a human-readable format that is difficult to parse programmatically.
• It has no support for runtime level control via external tools.
• It does not support tab-delimited structured output needed for centralized log aggregation.
• It cannot filter noisy third-party loggers (e.g., Jetty, SPI-Fly) that flood logs with low-value messages.
• It has no integration with Vespa's log rotation mechanism.

By installing a custom handler, Vespa gains full control over the log pipeline while maintaining compatibility with all libraries that use JUL.

Handler Architecture

The VespaLogHandler extends java.util.logging.StreamHandler and wraps the following components:

Component	Purpose
LogTarget	Provides the output stream (file, stderr, etc.) for each log write
LevelControllerRepo	Supplies per-component level controllers for shouldLog() checks
RejectFilter	Filters out known-noisy log messages that add no operational value
VespaFormatter	Formats log records into the tab-delimited Vespa log format
Level Reduction Map	Reduces the effective level of specific noisy loggers (e.g., Jetty INFO becomes DEBUG)

Installation Process

The handler installation occurs inside LogSetup.initInternal() and follows these steps:

1. Create the LogTarget: Parse the log target string (e.g., "fd:2", "file:/path/to/log") into a LogTarget implementation.
2. Create the LevelControllerRepo: Either a VespaLevelControllerRepo (with memory-mapped control file) or a DefaultLevelControllerRepo (level string only).
3. Construct the VespaLogHandler: Pass the log target, level controller repo, service name, and application prefix to the constructor.
4. Remove existing handlers: Remove all handlers currently attached to the JUL root logger.
5. Install the VespaLogHandler: Add the new handler to the JUL root logger.
6. Set the root logger level: Set the root logger level to ALL so that level filtering is handled by the VespaLogHandler rather than the JUL framework.

Level Reduction

Certain third-party libraries produce excessive log output at standard levels. The handler maintains a level reduction map that remaps specific loggers to lower levels:

// Example level reductions:
// org.eclipse.jetty: INFO -> DEBUG
// org.apache.aries.spifly: INFO -> DEBUG

This means that when a Jetty logger emits a message at INFO level, the handler treats it as DEBUG. Since DEBUG is off by default, this effectively silences noisy Jetty messages under normal operation while still allowing operators to see them by enabling DEBUG via vespa-logctl.

Reject Filtering

In addition to level control, the handler applies a reject filter that drops log messages matching known-useless patterns. This is a content-based filter that examines the log message text, unlike the level-based filtering which examines the log level.

The reject filter is created by RejectFilter.createDefaultRejectFilter() and includes patterns for messages that are known to be unhelpful in operations.

Thread Safety

The publish() method of VespaLogHandler is synchronized, ensuring that concurrent log calls from multiple threads produce well-formed, non-interleaved output lines. The synchronization also protects the open/write/close cycle of the log target.

Design Constraints

• Single handler per process: Only one VespaLogHandler should be installed on the root logger. Multiple handlers would produce duplicate output.
• Root logger attachment: The handler is attached to the JUL root logger, which means it captures log records from all JUL loggers in the process, including third-party library loggers.
• StreamHandler inheritance: By extending StreamHandler, the handler inherits standard formatting and output stream management, but overrides the stream lifecycle for log rotation support.

Interaction with Other Components

• Log Target and Level Configuration: Provides the log target and service name used to construct the handler.
• Control File Initialization: Provides the LevelControllerRepo for per-component level control.
• Log Message Formatting: The VespaFormatter installed in the handler converts records to Vespa's tab-delimited format.
• Log Rotation and Special Handling: The publish() method reopens the log target on each write to support external rotation.

References

• VespaLogHandler.java on GitHub
• Java StreamHandler API

Related Pages

Implemented By

• Implementation:Vespa_engine_Vespa_VespaLogHandler_Constructor