Implementation:Datahub project Datahub S3EmitterConfig

Knowledge Sources	Datahub_project_Datahub
Domains	Java_SDK, Metadata_Emission, AWS_S3
Last Updated	2026-02-10 00:00 GMT

Overview

An immutable Lombok-based configuration class that holds AWS S3 bucket, credential, and path settings for the S3Emitter.

Description

S3EmitterConfig is a Lombok @Value/@Builder class that encapsulates all settings required to construct an S3Emitter. It supports two credential strategies:

• Static credentials -- Provide accessKey and secretKey (and optionally sessionToken).
• Default credentials chain -- Omit static credentials and optionally specify a profileName and/or profileFile to customize AWS credential resolution.

The configuration also supports:

• Custom endpoints -- For S3-compatible storage services (e.g., MinIO, LocalStack) via the endpoint field.
• Path control -- pathPrefix defines the S3 key prefix, and fileName optionally overrides the auto-generated temporary file name.

Usage

Use this configuration class to set up S3 connection parameters before constructing an S3Emitter. The bucketName is the only required field; all other fields default to null or a Pegasus JSON EventFormatter.

Code Reference

Source Location

metadata-integration/java/datahub-client/src/main/java/datahub/client/s3/S3EmitterConfig.java

Signature

@Value
@Builder
public class S3EmitterConfig {
    @Builder.Default @NonNull String bucketName = null;
    @Builder.Default String pathPrefix = null;
    @Builder.Default String fileName = null;
    @Builder.Default EventFormatter eventFormatter =
        new EventFormatter(EventFormatter.Format.PEGASUS_JSON);
    @Builder.Default String region = null;
    @Builder.Default String endpoint = null;
    @Builder.Default String accessKey = null;
    @Builder.Default String secretKey = null;
    @Builder.Default String sessionToken = null;
    @Builder.Default String profileFile = null;
    @Builder.Default String profileName = null;
}

Import

import datahub.client.s3.S3EmitterConfig;

I/O Contract

Inputs

Field	Type	Default	Description
bucketName	String (@NonNull)	null (must be set)	S3 bucket name (required)
pathPrefix	String	null	Prefix for the S3 object key
fileName	String	null	Override name for the uploaded file; uses temp file name if null
eventFormatter	EventFormatter	Pegasus JSON	Formatter for MCP serialization
region	String	null	AWS region (e.g., "us-east-1")
endpoint	String	null	Custom S3 endpoint URL for S3-compatible services
accessKey	String	null	AWS access key ID for static credentials
secretKey	String	null	AWS secret access key for static credentials
sessionToken	String	null	AWS session token for temporary credentials
profileFile	String	null	Path to AWS credentials profile file
profileName	String	null	AWS profile name within the profile file

Outputs

An immutable S3EmitterConfig instance used to construct S3Emitter.

Usage Examples

// Minimal config with default AWS credentials chain
S3EmitterConfig config = S3EmitterConfig.builder()
    .bucketName("my-datahub-bucket")
    .pathPrefix("ingestion/")
    .build();

// With static credentials and custom endpoint
S3EmitterConfig config = S3EmitterConfig.builder()
    .bucketName("datahub")
    .pathPrefix("metadata")
    .fileName("batch_001.json")
    .endpoint("http://minio:9000")
    .accessKey("minioadmin")
    .secretKey("minioadmin")
    .region("us-east-1")
    .build();

// Using a named AWS profile
S3EmitterConfig config = S3EmitterConfig.builder()
    .bucketName("prod-datahub")
    .profileName("datahub-prod")
    .region("eu-west-1")
    .build();

Related Pages

• Datahub_project_Datahub_S3Emitter -- The emitter that consumes this configuration
• Datahub_project_Datahub_KafkaEmitterConfig -- Analogous configuration for the Kafka emitter
• Datahub_project_Datahub_RestEmitterConfig -- Analogous configuration for the REST emitter