Principle:Datahub project Datahub Sample Data Ingestion
| Field | Value |
|---|---|
| Page Type | Principle |
| Workflow | Docker_Quickstart_Deployment |
| Principle Name | Sample_Data_Ingestion |
| Repository | Datahub_project_Datahub |
| Implemented By | Implementation:Datahub_project_Datahub_Docker_CLI_Ingest_Sample_Data |
| Last Updated | 2026-02-09 17:00 GMT |
Overview
Description
Sample_Data_Ingestion is the principle of loading demonstration metadata into a metadata platform for evaluation and testing purposes. After deploying DataHub, the platform starts with an empty catalog. Without representative data, new users cannot evaluate the platform's search, lineage, governance, and discovery capabilities. Sample data ingestion bridges this gap by seeding the catalog with pre-built metadata entities that showcase the full range of platform features.
Usage
This principle is applied immediately after a successful quickstart deployment and health check verification. The sample data provides:
- Datasets -- Tables from multiple data platforms (e.g., Hive, HDFS, Kafka) with realistic schemas, descriptions, and tags.
- Lineage relationships -- Upstream/downstream connections between datasets that demonstrate lineage graph visualization.
- Ownership metadata -- Sample user entities associated with dataset ownership, enabling governance workflow exploration.
- Tags and glossary terms -- Pre-applied classification labels that demonstrate the tagging and glossary systems.
By loading this sample data, evaluators can explore a populated catalog within minutes of installation, without needing to configure real data source connections.
Theoretical Basis
Demo Data Seeding Pattern
The demo data seeding pattern is a well-established practice in software platforms, particularly those with complex data models. Its purpose is to reduce the time to first value for new users. Key characteristics of this pattern include:
- Self-contained -- The sample data does not depend on external systems. It is bundled with the platform or downloaded from a known location.
- Representative -- The data covers the major entity types and relationship patterns that the platform supports, providing a realistic preview of production usage.
- Idempotent -- Running the seeding operation multiple times produces the same result, avoiding duplicate entities.
- Non-destructive -- Sample data can coexist with real data without interfering with production workflows.
Rapid Platform Onboarding
The onboarding challenge for metadata platforms is that their value proposition -- discovery, lineage, governance -- can only be demonstrated with a populated catalog. An empty catalog is an abstract concept; a populated catalog is a tangible product. Sample data transforms the platform from the former into the latter.
This is particularly important for:
- Technical evaluations -- Engineering teams deciding whether to adopt DataHub need to see how it handles their use cases. Sample data lets them explore features without investing in real integrations first.
- Demos and presentations -- Pre-loaded data enables consistent, repeatable demonstrations of platform capabilities.
- Development and testing -- Developers working on DataHub itself need a populated instance to test UI changes, API modifications, and search behavior.
Ingestion Framework Integration
Sample data loading leverages the same ingestion framework used for production metadata extraction. This approach has two benefits:
- Consistency -- The sample data flows through the same code path as real metadata, ensuring that the demo accurately represents the platform's actual behavior.
- Educational value -- Observing the sample data ingestion process teaches new users how the ingestion framework works, preparing them to configure their own sources.
Related Pages
- Implementation:Datahub_project_Datahub_Docker_CLI_Ingest_Sample_Data -- The CLI command that performs sample data loading.
- Principle:Datahub_project_Datahub_Quickstart_Launch -- The deployment step that must complete before sample data can be loaded.
- Principle:Datahub_project_Datahub_UI_Access -- The web interface where loaded sample data becomes visible and explorable.
- Principle:Datahub_project_Datahub_Service_Health_Monitoring -- Health checks that must pass before ingestion can succeed.