Implementation:Apache Kafka Broker Log4j2 Configuration

Knowledge Sources	Apache_Kafka Kafka Operations Log4j2 Configuration
Domains	Operations, Logging, Configuration
Last Updated	2026-02-09 12:00 GMT

Overview

Configuration reference for the Kafka broker's primary Log4j2 logging configuration file (config/log4j2.yaml), defining appender hierarchies and logger routing for broker subsystems.

Description

The config/log4j2.yaml file is the central logging configuration for Kafka broker deployments. It defines two properties (kafka.logs.dir defaulting to . and logPattern set to [%d] %p %m (%c)%n), six RollingFile appenders with hourly time-based rotation, a Console appender (STDOUT), and a logger hierarchy that routes specific Kafka subsystems to dedicated log files using additivity: false to prevent duplicate logging.

Usage

This configuration is loaded by the Kafka broker at startup via the KAFKA_LOG4J_OPTS environment variable (set by kafka-server-start.sh). Operators customize this file to adjust log levels, add appenders, or change rotation policies for production Kafka clusters.

Code Reference

Source Location

• Repository: Apache_Kafka
• File: config/log4j2.yaml
• Lines: 1-162

Signature

Configuration:
  Properties:
    Property:
      - name: "kafka.logs.dir"
        value: "."
      - name: "logPattern"
        value: "[%d] %p %m (%c)%n"

  Appenders:
    Console:
      name: STDOUT
    RollingFile:
      - name: KafkaAppender          # -> server.log
      - name: StateChangeAppender     # -> state-change.log
      - name: RequestAppender         # -> kafka-request.log
      - name: CleanerAppender         # -> log-cleaner.log
      - name: ControllerAppender      # -> controller.log
      - name: AuthorizerAppender      # -> kafka-authorizer.log

  Loggers:
    Root:
      level: INFO
      AppenderRef: [STDOUT, KafkaAppender]
    Logger:
      - name: kafka.request.logger     -> RequestAppender (WARN)
      - name: kafka.network.RequestChannel$ -> RequestAppender (WARN)
      - name: org.apache.kafka.controller  -> ControllerAppender (INFO)
      - name: ...LogCleaner            -> CleanerAppender (INFO)
      - name: state.change.logger      -> StateChangeAppender (INFO)
      - name: kafka.authorizer.logger  -> AuthorizerAppender (INFO)

Import

# Applied automatically by kafka-server-start.sh via:
export KAFKA_LOG4J_OPTS="-Dlog4j2.configurationFile=$base_dir/../config/log4j2.yaml"

# Or override manually:
export KAFKA_LOG4J_OPTS="-Dlog4j2.configurationFile=/path/to/custom/log4j2.yaml"
bin/kafka-server-start.sh config/server.properties

I/O Contract

Inputs

Name	Type	Required	Description
kafka.logs.dir	System property	No	Directory for log file output; defaults to current directory (.)
KAFKA_LOG4J_OPTS	Env var	No	JVM system property pointing to this configuration file

Outputs

Name	Type	Description
server.log	Rolling file	General broker log output (Root logger + KafkaAppender)
state-change.log	Rolling file	Partition state transition events (StateChangeAppender)
kafka-request.log	Rolling file	Request handling logs at WARN+ level (RequestAppender)
log-cleaner.log	Rolling file	Log compaction activity (CleanerAppender)
controller.log	Rolling file	Controller operations (ControllerAppender)
kafka-authorizer.log	Rolling file	Authorization events (AuthorizerAppender)
STDOUT	Console	Console output for the Root logger

Usage Examples

Default Broker Startup

# The default config/log4j2.yaml is loaded automatically by kafka-server-start.sh
bin/kafka-server-start.sh config/server.properties
# Logs are written to the current directory (.) by default

Custom Log Directory

# Override the log directory via system property
export KAFKA_LOG4J_OPTS="-Dlog4j2.configurationFile=config/log4j2.yaml"
export LOG_DIR="/var/log/kafka"
bin/kafka-server-start.sh config/server.properties

Enable TRACE Request Logging

# Uncomment these sections in log4j2.yaml for detailed request tracing:
Logger:
  - name: kafka.network.Processor
    level: TRACE
    additivity: false
    AppenderRef:
      ref: RequestAppender
  - name: kafka.server.KafkaApis
    level: TRACE
    additivity: false
    AppenderRef:
      ref: RequestAppender

Related Pages

• Environment:Apache_Kafka_JVM_Runtime_Environment