Workflow:Datahub project Datahub Java SDK V2 Entity Management
| Knowledge Sources | |
|---|---|
| Domains | Data_Engineering, Metadata_Management, SDK, Java |
| Last Updated | 2026-02-09 12:00 GMT |
Overview
End-to-end process for managing DataHub metadata entities using the type-safe Java SDK V2 with fluent builders, patch-based updates, and lazy loading.
Description
This workflow covers the modern Java SDK V2 for DataHub, which provides a high-level, entity-oriented API for creating and managing metadata. Unlike the V1 SDK that required manual MCP construction, V2 uses fluent builder patterns for compile-time type safety, supports efficient patch-based updates (JSON Patch RFC 6902), and includes lazy loading with aspect caching. Supported entities include Dataset, DataFlow, DataJob, Chart, Dashboard, Container, MLModel, and MLModelGroup.
Usage
Execute this workflow when building Java applications that need to create, read, update, or delete metadata entities in DataHub. This is the recommended approach for new Java projects. Use it for CI/CD metadata emission, custom data pipeline orchestrators, or any Java service that manages data catalog metadata.
Execution Steps
Step 1: Add SDK Dependency
Add the datahub-client library to your Java project build configuration (Gradle or Maven). The SDK is published to Maven Central.
Key considerations:
- Use the latest version from Maven Central
- The dependency includes all supported entity types
- Compatible with Java 8+
Step 2: Initialize Client
Create a DataHubClientV2 instance using the builder pattern, providing the DataHub server URL and authentication token.
Key considerations:
- Server URL points to the GMS REST endpoint
- Authentication token is optional for local development
- Client is thread-safe and should be reused across operations
- Client should be closed when no longer needed to release resources
Step 3: Build Entity
Use the entity-specific fluent builder to create a new entity instance with its key properties (platform, name, environment).
Key considerations:
- Each entity type has its own builder (Dataset.builder(), Chart.builder(), etc.)
- Key properties define the entity's URN (unique identity)
- Builders enforce required fields at compile time
Step 4: Add Metadata
Enrich the entity with metadata using chainable methods for tags, owners, custom properties, glossary terms, structured properties, and domain assignments.
Key considerations:
- Methods like addTag(), addOwner(), addCustomProperty() are chainable
- Schema fields can be added for datasets
- Lineage relationships are set via dedicated methods
- Metadata changes accumulate locally until saved
Step 5: Upsert Entity
Persist the entity to DataHub by calling the client upsert method. This creates or updates the entity, emitting only changed aspects.
Key considerations:
- Upsert creates a new entity or updates all aspects of an existing one
- Update uses patch semantics for incremental changes
- Emit modes control write acknowledgment: SYNC_WAIT, SYNC_PRIMARY, ASYNC
- Version conflicts can be detected and handled
Step 6: Read and Modify Entities
Fetch existing entities from DataHub for inspection or modification. Fetched entities are read-only; call mutableCopy() to create a writable version.
Key considerations:
- client.entities().get(urn) fetches an entity
- Fetched entities are immutable by default for safety
- mutableCopy() creates a writable clone for modifications
- Aspects are loaded lazily on first access