Heuristic:Apache Kafka Log4j Migration Compatibility
| Knowledge Sources | |
|---|---|
| Domains | Debugging, Configuration |
| Last Updated | 2026-02-09 12:00 GMT |
Overview
Kafka auto-detects legacy Log4j 1.x configuration files and enables a compatibility bridge, but operators should migrate to Log4j 2.x YAML format to avoid deprecation.
Description
Apache Kafka has migrated from Log4j 1.x to Log4j 2.x. The `kafka-run-class.sh` launcher script detects whether the user's `KAFKA_LOG4J_OPTS` points to a Log4j 1.x configuration file (matching patterns `log4j.*.properties` or `log4j.*.xml`) and, if so, automatically sets the `LOG4J_COMPATIBILITY=true` environment variable. This enables Log4j 2's built-in Log4j 1.x configuration compatibility mode. The script also prints a deprecation warning to stderr directing the operator to migrate. The default configuration now uses `tools-log4j2.yaml` (YAML format) for tools and `log4j2.yaml` for the broker.
Usage
Apply this heuristic when upgrading Kafka and encountering logging configuration issues, or when deploying Kafka with custom Log4j configurations. If you see the deprecation warning, plan a migration to the Log4j 2.x YAML configuration format.
The Insight (Rule of Thumb)
- Action: Migrate Log4j 1.x `.properties` or `.xml` configs to Log4j 2.x YAML format.
- Value: Use `$KAFKA_HOME/config/log4j2.yaml` as a starting template.
- Trade-off: Compatibility mode works but is deprecated; may not support all Log4j 2.x features.
- Detection: The script pattern-matches `log4j.*.(properties|xml)` in `KAFKA_LOG4J_OPTS`.
Reasoning
Log4j 1.x reached end-of-life and has known security vulnerabilities. Kafka migrated to Log4j 2.x, which uses a different configuration format (YAML/JSON/XML with different schema). To avoid breaking existing deployments that use custom Log4j 1.x configs, the launcher script detects the old format and enables a compatibility bridge. However, this bridge is a shim and does not support all Log4j 2.x features. Operators should migrate to the native Log4j 2.x format to get full functionality and avoid the deprecation path.
Code evidence from `bin/kafka-run-class.sh:220-234`:
# Log4j settings
if [ -z "$KAFKA_LOG4J_OPTS" ]; then
# Log to console. This is a tool.
LOG4J_DIR="$base_dir/config/tools-log4j2.yaml"
KAFKA_LOG4J_OPTS="-Dlog4j2.configurationFile=${LOG4J_DIR}"
else
if echo "$KAFKA_LOG4J_OPTS" | grep -E "log4j\.[^[:space:]]+(\.properties|\.xml)$" >/dev/null; then
# Enable Log4j 1.x configuration compatibility mode for Log4j 2
export LOG4J_COMPATIBILITY=true
echo DEPRECATED: A Log4j 1.x configuration file has been detected, which is no longer recommended. >&2
echo To use a Log4j 2.x configuration, please see https://logging.apache.org/log4j/2.x/migrate-from-log4j1.html ... >&2
echo You can also use the \$KAFKA_HOME/config/tools-log4j2.yaml file as a starting point. >&2
fi
fi
Broker default from `bin/kafka-server-start.sh:24-26`:
if [ -z "$KAFKA_LOG4J_OPTS" ]; then
export KAFKA_LOG4J_OPTS="-Dlog4j2.configurationFile=$base_dir/../config/log4j2.yaml"
fi