Implementation:Apache Airflow PercentFormatRender

Knowledge Sources	Apache_Airflow
Domains	Logging, Formatting
Last Updated	2026-02-08 21:00 GMT

Overview

A structlog processor that renders structured log events using Python's %-style (percent) format strings, providing backward compatibility with legacy Airflow log format configurations and supporting color output, callsite information, and extra event fields.

Description

The PercentFormatRender class extends structlog's ConsoleRenderer to support the traditional %(name)s-style format strings that were used in Airflow's stdlib logging configuration. This bridge allows Airflow to use structlog internally while still honoring user-configured log format strings.

Key design aspects:

• Lazy log record dict: The _LazyLogRecordDict helper class implements collections.abc.Mapping and translates structlog event dict keys to their stdlib logging.LogRecord equivalents on demand. This avoids materializing a full LogRecord object while supporting format strings that reference fields like %(name)s, %(levelname)s, %(asctime)s, %(message)s, %(filename)s, %(lineno)d, etc.

• Callsite parameter mapping: The class maintains a callsite_parameters class variable that maps stdlib log record attribute names (e.g., pathname, filename, module, lineno, funcName, thread, threadName, process, processName) to structlog's CallsiteParameter enum values.

• Color support: The lazy dict supports color placeholder keys (%(red)s, %(green)s, %(blue)s, %(log_color)s, %(reset)s, etc.) for use with colorized log format strings. When colors are disabled, these return empty strings.

• Extra fields: After rendering the format string, any remaining keys in the event dict that are not part of the special_keys set are appended as space-separated key=value pairs.

• Exception handling: The processor properly handles exception, exc_info, and stack keys from the event dict, formatting them after the main log line.

Usage

from airflow_shared.logging.percent_formatter import PercentFormatRender

# Create a processor with a traditional log format string
processor = PercentFormatRender(
    fmt="[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s"
)

# Use as a structlog processor in a processor chain
import structlog
structlog.configure(
    processors=[
        structlog.stdlib.add_log_level,
        structlog.processors.CallsiteParameterAdder(
            PercentFormatRender.callsite_params_from_fmt_string(
                "[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s"
            )
        ),
        processor,
    ],
)

Code Reference

Source Location

• Repository: Apache_Airflow
• File: shared/logging/src/airflow_shared/logging/percent_formatter.py

Key Classes

PercentFormatRender (lines 97-188)

class PercentFormatRender(ConsoleRenderer):
    """A Structlog processor that uses a stdlib-like percent based format string."""

    _fmt: str

    callsite_parameters: ClassVar[dict[str, CallsiteParameter]] = {
        "pathname": CallsiteParameter.PATHNAME,
        "filename": CallsiteParameter.FILENAME,
        "module": CallsiteParameter.MODULE,
        "lineno": CallsiteParameter.LINENO,
        "funcName": CallsiteParameter.FUNC_NAME,
        "thread": CallsiteParameter.THREAD,
        "threadName": CallsiteParameter.THREAD_NAME,
        "process": CallsiteParameter.PROCESS,
        "processName": CallsiteParameter.PROCESS_NAME,
    }

    special_keys = {
        "event", "name", "logger", "logger_name", "timestamp", "level",
    } | set(map(operator.attrgetter("value"), callsite_parameters.values()))

    @classmethod
    def callsite_params_from_fmt_string(
        cls, fmt: str,
    ) -> collections.abc.Iterable[CallsiteParameter]:
        """Extract required CallsiteParameter values from a format string."""

    def __init__(self, fmt: str, **kwargs): ...

    def __call__(
        self, logger: WrappedLogger, method_name: str, event_dict: EventDict,
    ) -> str:
        """Render the event dict using the percent-style format string."""

_LazyLogRecordDict (lines 37-94)

class _LazyLogRecordDict(collections.abc.Mapping):
    """Lazy mapping that translates structlog event dict keys to stdlib LogRecord equivalents."""

    __slots__ = ("event", "styles", "level_styles", "method_name", "no_colors")

    def __init__(
        self, event: EventDict, method_name: str,
        level_styles: dict[str, str], styles: ColumnStyles,
    ): ...

    def __getitem__(self, key):
        """Translate keys: name, levelname, asctime, message, lineno,
        filename, funcName, color names, log_color, reset, etc."""

    def __iter__(self): ...
    def __len__(self): ...

Import

from airflow_shared.logging.percent_formatter import PercentFormatRender

I/O Contract

Inputs

Name	Type	Required	Description
fmt	str	Yes (constructor)	A Python %-style format string (e.g., "%(levelname)s - %(message)s")
logger	WrappedLogger	Yes (per call)	The structlog wrapped logger instance
method_name	str	Yes (per call)	The log method name (e.g., "info", "error")
event_dict	EventDict	Yes (per call)	The structlog event dictionary containing log data

Outputs

Name	Type	Description
Formatted string	str	The fully rendered log line, including the format string output, extra fields, and any exception/stack information
CallsiteParameter iterable	Iterable[CallsiteParameter]	From callsite_params_from_fmt_string(): the set of callsite parameters needed by the format string

Supported Format Keys

Format Key	Source	Description
%(name)s	event_dict["logger"] or event_dict["logger_name"]	Logger name
%(levelname)s	event_dict["level"] (uppercased)	Log level name
%(asctime)s / %(created)s	event_dict["timestamp"] or current UTC ISO time	Timestamp
%(message)s	event_dict["event"]	The log message
%(filename)s	Callsite info or "(unknown file)"	Source filename
%(lineno)d	Callsite info or 0	Source line number
%(funcName)s	Callsite info or "(unknown function)"	Function name
%(pathname)s	CallsiteParameter.PATHNAME	Full source path
%(module)s	CallsiteParameter.MODULE	Module name
%(thread)d	CallsiteParameter.THREAD	Thread ID
%(threadName)s	CallsiteParameter.THREAD_NAME	Thread name
%(process)d	CallsiteParameter.PROCESS	Process ID
%(processName)s	CallsiteParameter.PROCESS_NAME	Process name
%(log_color)s	Level-based ANSI color	Color code for the current log level
%(reset)s	ANSI reset code	Resets color output
%(red)s, %(green)s, etc.	ANSI color codes	Named color codes (empty when colors disabled)

Usage Examples

Airflow Default Log Format

from airflow_shared.logging.percent_formatter import PercentFormatRender

# Airflow's default log format
LOG_FORMAT = "[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s"

renderer = PercentFormatRender(fmt=LOG_FORMAT)
# When called as a structlog processor, produces output like:
# [2024-06-15T10:30:00+00:00] {my_dag.py:42} INFO - Task execution started

Extracting Required Callsite Parameters

from airflow_shared.logging.percent_formatter import PercentFormatRender
from structlog.processors import CallsiteParameterAdder

fmt = "[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s"
params = list(PercentFormatRender.callsite_params_from_fmt_string(fmt))
# params = [CallsiteParameter.FILENAME, CallsiteParameter.LINENO]

# Use with structlog's CallsiteParameterAdder
adder = CallsiteParameterAdder(parameters=params)

Colorized Log Format

from airflow_shared.logging.percent_formatter import PercentFormatRender

COLOR_FORMAT = "%(log_color)s%(levelname)s%(reset)s - %(blue)s%(name)s%(reset)s - %(message)s"
renderer = PercentFormatRender(fmt=COLOR_FORMAT)
# Produces ANSI-colored output when colors are enabled

Related Pages

• Environment:Apache_Airflow_Python_Runtime_Environment