Jump to content

Connect Leeroopedia MCP: Equip your AI agents to search best practices, build plans, verify code, diagnose failures, and look up hyperparameter defaults.

Environment:Apache Kafka JVM Runtime Environment

From Leeroopedia


Knowledge Sources
Domains Infrastructure, JVM
Last Updated 2026-02-09 12:00 GMT

Overview

JDK 11+ runtime for Kafka client modules, JDK 17+ for broker and server modules, with Scala 2.13.18 on the classpath.

Description

Apache Kafka requires a Java Development Kit (JDK) to build and run. The codebase defines two minimum Java version tiers: Java 11 for client-facing modules (clients, streams, generator) and Java 17 for all non-client modules (broker, coordinator, server components). The runtime classpath is assembled by `kafka-run-class.sh`, which detects the installed Java version and adapts JVM flags accordingly (e.g., GC logging format differs between Java 8 and Java 9+). Docker images use Eclipse Temurin JDK 21 (Alpine JRE variant) as the production runtime.

Usage

Use this environment for any Kafka operation: building from source, running brokers, executing CLI tools (kafka-topics, kafka-configs), or running integration tests. All shell scripts in `bin/` delegate to `kafka-run-class.sh`, which assembles the classpath and launches the JVM.

System Requirements

Category Requirement Notes
OS Linux, macOS, Windows (Cygwin/MinGW) Cygwin/MinGW paths auto-converted
Java (Clients) JDK 11+ Modules: clients, generator, streams, streams:test-utils, streams:examples, streams:streams-scala, test-common:test-common-util
Java (Server) JDK 17+ All non-client modules (broker, coordinator, tools, etc.)
Java (Docker) Eclipse Temurin 21 JRE Alpine Production Docker images use JDK 21
Scala 2.13.18 Default version; read from gradle.properties
Disk Varies Log directories created under `$LOG_DIR` or `$base_dir/logs`

Dependencies

System Packages

  • `java` >= 11 (clients) / >= 17 (server)
  • `bash` (required by all shell scripts)

JVM Classpath

  • Kafka core JARs (assembled from `libs/` directory)
  • Scala library 2.13.18
  • Log4j 2.x configuration (YAML format)

Credentials

The following environment variables may be set:

  • `JAVA_HOME`: Path to JDK installation (optional; falls back to `java` on PATH).
  • `KAFKA_HEAP_OPTS`: JVM heap options (default: `-Xmx256M` for tools, `-Xmx1G -Xms1G` for broker).
  • `KAFKA_JVM_PERFORMANCE_OPTS`: JVM performance flags (default: G1GC with tuned parameters).
  • `KAFKA_LOG4J_OPTS`: Path to Log4j configuration file.
  • `KAFKA_OPTS`: Additional JVM options passed to all Kafka processes.
  • `KAFKA_DEBUG`: Set to enable remote debugging (default port 5005).
  • `JAVA_DEBUG_PORT`: Remote debug port (default: 5005).
  • `JMX_PORT`: JMX monitoring port.
  • `KAFKA_JMX_OPTS`: JMX configuration flags.
  • `LOG_DIR`: Log output directory.
  • `GC_LOG_ENABLED`: Enable GC logging (set via `-loggc` flag).
  • `KAFKA_MODE`: Set to `native` for GraalVM native image execution.
  • `SCALA_VERSION`: Override Scala version (default: 2.13.18).

Quick Install

# Verify Java version (must be 17+ for server modules)
java -version

# Build Kafka from source
./gradlew jar

# Or for a full release build
./gradlew releaseTarGz

Code Evidence

Java version tiers from `build.gradle:47-49`: 

ext {
  minClientJavaVersion = 11
  minNonClientJavaVersion = 17
  modulesNeedingJava11 = [":clients", ":generator", ":streams", ":streams:test-utils",
    ":streams:examples", ":streams:streams-scala", ":test-common:test-common-util"]
}

Java 16+ compatibility workaround from `build.gradle:56-72`: 

// "JEP 403: Strongly Encapsulate JDK Internals" causes some tests to fail
// when they try to access internals (often via mocking libraries).
// We use `--add-opens` as a workaround for now via KAFKA-13275.
if (JavaVersion.current().isCompatibleWith(JavaVersion.VERSION_16))
  defaultJvmArgs.addAll(
    "--add-opens=java.base/java.io=ALL-UNNAMED",
    "--add-opens=java.base/java.lang=ALL-UNNAMED",
    "--add-opens=java.base/java.nio=ALL-UNNAMED",
    // ... additional modules
  )

Java version detection from `bin/kafka-run-class.sh:318-331`: 

JAVA_MAJOR_VERSION=$("$JAVA" -version 2>&1 | sed -E -n 's/.* version "([0-9]*).*$/\1/p')
if [[ "$JAVA_MAJOR_VERSION" -ge "9" ]] ; then
  KAFKA_GC_LOG_OPTS="-Xlog:gc*:file=$LOG_DIR/$GC_LOG_FILE_NAME:time,tags:filecount=10,filesize=100M"
else
  KAFKA_GC_LOG_OPTS="-Xloggc:$LOG_DIR/$GC_LOG_FILE_NAME -verbose:gc -XX:+PrintGCDetails ..."
fi

Common Errors

Error Message Cause Solution
`JAVA_HOME is not set and java could not be found in PATH` Java not installed or not on PATH Install JDK 17+ and set `JAVA_HOME` or add `java` to `PATH`
`UnsupportedClassVersionError` Running server modules with JDK < 17 Upgrade to JDK 17 or later
`java.lang.reflect.InaccessibleObjectException` JDK 16+ strong encapsulation Add `--add-opens` flags (done automatically by build.gradle for tests)
`DEPRECATED: A Log4j 1.x configuration file has been detected` Legacy Log4j 1.x config detected Migrate to Log4j 2.x YAML config format

Compatibility Notes

  • Java 11 vs 17: Client modules (clients, streams) compile to Java 11 target for maximum compatibility. Server modules require Java 17.
  • Java 16+ (JEP 403): Tests requiring JDK internals need `--add-opens` flags; these are auto-configured in `build.gradle`.
  • GC Logging: Java 9+ uses `-Xlog:gc*` unified logging; Java 8 uses legacy `-Xloggc:` format.
  • Windows: Cygwin and MinGW detected automatically; classpath converted to Windows format.
  • Native Mode: Set `KAFKA_MODE=native` to run GraalVM native image (requires pre-built binary at `$base_dir/kafka.Kafka`).

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment
Retrieved from "https://leeroopedia.com/index.php?title=Environment:Apache_Kafka_JVM_Runtime_Environment&oldid=27495"