Implementation:Deepset ai Haystack Logging Configuration
| Knowledge Sources | |
|---|---|
| Domains | Logging, Infrastructure, Observability |
| Last Updated | 2026-02-11 20:00 GMT |
Overview
Concrete tool for configuring structured logging with keyword-argument enforcement and optional structlog integration provided by the Haystack framework.
Description
The haystack.logging module provides a custom structured logging system built on top of Python's standard logging library. It enforces keyword-only arguments for all log methods (debug, info, warning, error, etc.) to ensure consistency and make structured logging effective by default. The module patches standard logger methods via decorators (patch_log_method_to_kwargs_only, patch_log_with_level_method_to_kwargs_only) and uses str.format() interpolation through the extra dict via patch_make_records_to_use_kwarg_string_interpolation.
When structlog is installed, configure_logging() sets up shared processors for timestamps, log levels, line/file information, and trace correlation. It supports both console rendering (for interactive terminals) and JSON output (for production), auto-detecting the mode based on TTY status or the HAYSTACK_LOGGING_USE_JSON environment variable. The HAYSTACK_LOGGING_IGNORE_STRUCTLOG environment variable allows users to disable structlog integration entirely.
The PatchedLogger Protocol class defines the expected interface for type checking, ensuring correct logger usage across the codebase.
Usage
Use haystack.logging.getLogger(name) instead of logging.getLogger(name) when creating loggers within Haystack components. Call configure_logging() to set up structured logging with structlog integration. This is called automatically on import haystack. Use HAYSTACK_LOGGING_USE_JSON=true for production JSON output, or HAYSTACK_LOGGING_IGNORE_STRUCTLOG=true to fall back to standard logging.
Code Reference
Source Location
- Repository: Deepset_ai_Haystack
- File: haystack/logging.py
- Lines: 1-404
Signature
class PatchedLogger(typing.Protocol):
"""Class which enables using type checkers to find wrong logger usage."""
def debug(self, msg: str, *, _: Any = None, exc_info: Any = None,
stack_info: Any = False, stacklevel: int = 1, **kwargs: Any) -> None: ...
def info(self, msg: str, *, **kwargs: Any) -> None: ...
def warning(self, msg: str, *, **kwargs: Any) -> None: ...
def error(self, msg: str, *, **kwargs: Any) -> None: ...
def critical(self, msg: str, *, **kwargs: Any) -> None: ...
def exception(self, msg: str, *, **kwargs: Any) -> None: ...
def log(self, level: int, msg: str, *, **kwargs: Any) -> None: ...
def setLevel(self, level: int) -> None: ...
def getLogger(name: str) -> PatchedLogger:
"""Get the Haystack logger, a patched version of the standard library logger."""
...
def configure_logging(use_json: bool | None = None) -> None:
"""Configure logging for Haystack with optional structlog integration."""
...
def correlate_logs_with_traces(
_: "WrappedLogger", __: str, event_dict: "EventDict"
) -> "EventDict":
"""Add correlation data for logs to enable trace correlation."""
...
Import
from haystack.logging import getLogger, configure_logging
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| name | str | Yes | Logger name, typically the module's __name__ |
| use_json | bool or None | No | Force JSON output (True), console output (False), or auto-detect (None) |
Outputs
| Name | Type | Description |
|---|---|---|
| getLogger returns | PatchedLogger | A patched logger that enforces keyword-only arguments |
| configure_logging returns | None | Configures structlog formatting on the root logger |
Usage Examples
Basic Logger Usage
from haystack.logging import getLogger
logger = getLogger(__name__)
# Keyword arguments are enforced - values go into structured 'extra' dict
logger.info("Processing document {doc_id} with {num_words} words", doc_id="abc123", num_words=500)
# Standard positional args are NOT allowed (will raise TypeError)
# logger.info("Processing %s", "abc123") # WRONG
Configuring JSON Logging for Production
from haystack.logging import configure_logging
# Force JSON output regardless of terminal detection
configure_logging(use_json=True)
Environment Variable Configuration
# Enable JSON logging via environment variable
export HAYSTACK_LOGGING_USE_JSON=true
# Disable structlog integration entirely
export HAYSTACK_LOGGING_IGNORE_STRUCTLOG=true