Implementation:Datahub project Datahub Sample BQ Data

Knowledge Sources	Datahub_project_Datahub
Domains	Testing
Last Updated	2026-02-10 00:00 GMT

Overview

A sample JSON data file containing Metadata Change Events (MCEs) for BigQuery datasets, used as test fixture data for DataHub smoke tests to verify ingestion, search, and metadata operations.

Description

sample_bq_data.json is a JSON array of MCE (Metadata Change Event) objects in the legacy snapshot format. Each element contains a proposedSnapshot with entity-specific aspects. The file includes the following entities:

Dataset: bigquery-public-data.covid19_geotab_mobility_impact.us_border_wait_times

• DatasetProperties -- Description of US border wait times data (includes a Korean character for Unicode testing)
• SchemaMetadata -- 20 schema fields including:
  • border_id, port_name, port_name_us (StringType)
  • hour_local, hour_utc, avg_crossing_duration, percent_of_baseline_trip_volume, border_latitude, border_longitude (NumberType)
  • date_local, date_utc (DateType)
  • trip_direction, day_type, aggregation_method, province_code, state_code_us, border_geohash, version (StringType)
  • border_zone, border_zone_us (NullType -- geography polygons)
• Ownership -- Owned by urn:li:corpuser:Geotab as DATAOWNER

CorpUser: urn:li:corpuser:Geotab

• CorpUserInfo -- Active user with display name "Geotab" and demo email

Dataset: bigquery-public-data.covid19_geotab_mobility_impact.ca_border_wait_times

• DatasetProperties -- Description of Canada-US border wait times

The schema fields demonstrate various DataHub type mappings: StringType, NumberType, DateType, and NullType. All fields are marked as nullable with "recursive": false. The platform is BigQuery with PROD fabric type.

Usage

This file is ingested during smoke test setup using ingest_file_via_rest() to populate the DataHub instance with known test data. Tests then query this data to verify search, browse, schema rendering, ownership, and other metadata operations. It is cleaned up after test completion via delete_urns_from_file().

Code Reference

Source Location

• Repository: Datahub_project_Datahub
• File: smoke-test/sample_bq_data.json

Signature

[
    {
        "auditHeader": null,
        "proposedSnapshot": {
            "com.linkedin.pegasus2avro.metadata.snapshot.DatasetSnapshot": {
                "urn": "urn:li:dataset:(urn:li:dataPlatform:bigquery,...,PROD)",
                "aspects": [
                    {"com.linkedin.pegasus2avro.dataset.DatasetProperties": {...}},
                    {"com.linkedin.pegasus2avro.schema.SchemaMetadata": {...}}
                ]
            }
        },
        "proposedDelta": null
    }
]

Import

# Used in smoke tests via the ingestion helper
from tests.utils import ingest_file_via_rest

ingest_file_via_rest(auth_session, "sample_bq_data.json")

I/O Contract

Input	Type	Description
(none -- this is a data file)		Ingested via REST API by smoke test fixtures

Output	Type	Description
Dataset entities	DataHub entities	Two BigQuery dataset entities with schema and properties
CorpUser entity	DataHub entity	One corporate user entity (Geotab)
Ownership aspect	DataHub aspect	Ownership association between dataset and user

Entities in File

Entity URN	Entity Type	Aspects
urn:li:dataset:(urn:li:dataPlatform:bigquery,bigquery-public-data.covid19_geotab_mobility_impact.us_border_wait_times,PROD)	Dataset	DatasetProperties, SchemaMetadata, Ownership
urn:li:corpuser:Geotab	CorpUser	CorpUserInfo
urn:li:dataset:(urn:li:dataPlatform:bigquery,bigquery-public-data.covid19_geotab_mobility_impact.ca_border_wait_times,PROD)	Dataset	DatasetProperties