Environment:Datahub project Datahub Java 17 Backend Environment

Knowledge Sources	DataHub build.gradle DataHub Developer Guide
Domains	Infrastructure, Java, Backend
Last Updated	2026-02-10 00:00 GMT

Overview

Java 17 (JDK) build and runtime environment with Gradle 8.14, Spring Boot 3.4, and Kafka 8.0 for building and running DataHub backend services and Java SDK V2.

Description

This environment defines the Java toolchain required for building the DataHub backend (GMS), Java SDK V2 client library, and associated modules. JDK 17 is the default and is enforced for all Spring 6.x / Spring Boot 3.x modules. The bytecode target defaults to Java 11 for broader compatibility, but Spring modules compile to Java 17 bytecode. The Spark lineage module can optionally target Java 8 bytecode via `-PjavaClassVersionDefault=8` for Spark cluster compatibility.

Usage

Use this environment for building DataHub from source, developing Java SDK V2 integrations, or running the GMS backend service. It is required for all Gradle build commands (`./gradlew build`, `./gradlew check`, `./gradlew spotlessApply`).

System Requirements

Category	Requirement	Notes
OS	Linux, macOS, Windows	Any OS supported by JDK 17
JDK	Java 17 (OpenJDK or Oracle JDK)	Hard requirement for Spring Boot 3.x modules
Build Tool	Gradle 8.14.3	Bundled via Gradle Wrapper (`./gradlew`)
RAM	4GB+	For Gradle builds; more for parallel compilation
Disk	10GB+	Build artifacts, Gradle cache, and dependencies

Dependencies

Core Framework Versions

`Spring Framework` = 6.2.11
`Spring Boot` = 3.4.5
`Spring Kafka` = 3.3.8
`Pegasus (rest.li)` = 29.74.2

Key Library Versions

`Elasticsearch client` = 2.19.4 (for ES 7.10, OpenSearch 1.x, 2.x)
`Elasticsearch 8 client` = 8.17.4
`Neo4j` = 5.20.0
`Kafka (Confluent)` = 8.0.0
`Jackson` = 2.18.4
`Ebean ORM` = 15.5.2
`Hadoop` = 3.3.6
`OpenLineage` = 1.33.0
`Hazelcast` = 5.3.6
`Jetty` = 12.0.21
`AWS SDK v2` = 2.30.33

Test Dependencies

`JUnit Jupiter` = 5.6.1
`TestContainers` = 1.21.1

Credentials

No credentials required for building. Runtime credentials depend on deployment configuration:

`DATAHUB_GMS_TOKEN`: GMS authentication token (for Java SDK V2 client)
`KAFKA_BOOTSTRAP_SERVER`: Kafka bootstrap server address
`ELASTICSEARCH_HOST`: Elasticsearch host URL
`MYSQL_HOST` / `MYSQL_PORT`: Database connection details

Quick Install

# Verify Java version
java -version  # Must show 17.x

# Build entire project
./gradlew build

# Run tests and linting
./gradlew check

# Format Java code
./gradlew spotlessApply

# Build Spark lineage JAR with Java 8 bytecode target
./gradlew -PjavaClassVersionDefault=8 :metadata-integration:java:acryl-spark-lineage:shadowJar

Code Evidence

JDK version defaults from `build.gradle:5-6`:

ext.jdkVersionDefault = 17
ext.javaClassVersionDefault = 11

Spring 6 hard dependency on JDK 17 from `build.gradle:10-21`:

ext.jdkVersion = { p ->
    // If Spring 6 is present, hard dependency on jdk17
    if (p.configurations.any { it.getDependencies().any{
      (it.getGroup().equals("org.springframework") && it.getVersion().startsWith("6."))
      || (it.getGroup().equals("org.springframework.boot") && it.getVersion().startsWith("3."))
    }} || springModules.contains(p.name)) {
      return 17
    } else {
      return p.hasProperty('jdkVersionDefault') ? Integer.valueOf(...) : ext.jdkVersionDefault
    }
}

Framework versions from `build.gradle:36-67`:

ext.springVersion = '6.2.11'
ext.springBootVersion = '3.4.5'
ext.springKafkaVersion = '3.3.8'
ext.kafkaVersion = '8.0.0'
ext.neo4jVersion = '5.20.0'
ext.elasticsearchVersion = '2.19.4'

Common Errors

Error Message	Cause	Solution
`Unsupported class file major version 61`	JDK version mismatch (Java 17 bytecode on older JVM)	Ensure JDK 17 is active: `java -version`
`Could not determine java version from '21'`	JDK 21 not fully supported by Gradle wrapper	Use JDK 17 specifically
Gradle OOM during build	Insufficient heap for Gradle daemon	Set `org.gradle.jvmargs=-Xmx4g` in `gradle.properties`
`generateGitPropertiesGlobal` failure	Git worktree environment issue	Add `-x generateGitPropertiesGlobal` to Gradle command

Compatibility Notes

Git worktrees: When using git worktrees, exclude the git properties task: `./gradlew build -x generateGitPropertiesGlobal`
Spark lineage: The Spark lineage module can target Java 8 bytecode via `-PjavaClassVersionDefault=8` for older Spark clusters.
Akka version: Pinned at 2.6.21 because 2.7.0+ has an incompatible license (BSL).
Play Framework: Uses Play 2.8.22 with Scala 2.13 for the frontend proxy module.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment