Implementation:Mage ai Mage ai Twitter Ads Schema

Knowledge Sources	Mage Integrations Twitter Ads API
Domains	Data_Integration, Twitter_Ads, Schema
Last Updated	2026-02-09 00:00 GMT

Overview

Schema loading and metadata generation module for the Mage Twitter Ads source connector, responsible for loading JSON schemas, resolving shared references, generating Singer metadata, and validating report configurations.

Description

The get_schemas function is the primary entry point of this module. It performs two main tasks:

1. Standard stream schemas - For each stream defined in STREAMS, loads the corresponding JSON schema file from the schemas/ directory, resolves $ref references using shared schemas, generates standard Singer metadata with key properties and replication keys, and marks replication keys with inclusion: automatic.

2. Report schemas - For each report in the config, validates the report configuration against allowed ENTITY_TYPES (8 types including ACCOUNT, CAMPAIGN, LINE_ITEM, etc.), SEGMENTS (23 types including NO_SEGMENT, AGE, DEVICES, LOCATIONS, etc.), and GRANULARITIES (HOUR, DAY, TOTAL). Applies business rules for invalid combinations (e.g., MEDIA_CREATIVE cannot have segments, CONVERSION_TAGS restricted to certain entities). Selects the appropriate report schema template (report_web_conversion, report_{entity}, or report_other). Removes segment fields for NO_SEGMENT and web_conversion fields for incompatible segments.

Helper functions include:

• load_shared_schema_refs() - Loads all JSON files from schemas/shared/ directory.
• make_replication_key_automatic(mdata, schema, replication_keys) - Sets inclusion: automatic for replication key fields in metadata.

Usage

Called during the discovery phase to build schemas and metadata for the Singer catalog.

Code Reference

Source Location

• Repository: mage-ai
• File: mage_integrations/mage_integrations/sources/twitter_ads/tap_twitter_ads/schema.py
• Lines: 1-224

Signature

GRANULARITIES = ['HOUR', 'DAY', 'TOTAL']
ENTITY_TYPES = ['ACCOUNT', 'CAMPAIGN', 'FUNDING_INSTRUMENT', 'LINE_ITEM',
                'MEDIA_CREATIVE', 'ORGANIC_TWEET', 'PROMOTED_TWEET', 'PROMOTED_ACCOUNT']
SEGMENTS = ['NO_SEGMENT', 'AGE', 'AUDIENCES', 'DEVICES', 'GENDER', ...]

def get_abs_path(path):
def load_shared_schema_refs():
def make_replication_key_automatic(mdata, schema, replication_keys):
def get_schemas(reports, logger=LOGGER):

Import

from mage_integrations.sources.twitter_ads.tap_twitter_ads.schema import get_schemas

I/O Contract

Inputs

Name	Type	Required	Description
reports	list[dict]	Yes	List of report configurations with name, entity, segment, granularity
logger	Logger	No	Optional logger (defaults to Singer LOGGER)

Outputs

Name	Type	Description
schemas	dict	Mapping of stream/report names to JSON schema dictionaries
field_metadata	dict	Mapping of stream/report names to Singer metadata lists

Validation Rules

Rule	Description
Entity validation	Must be one of 8 predefined ENTITY_TYPES
Segment validation	Must be one of 23 predefined SEGMENTS
Granularity validation	Must be HOUR, DAY, or TOTAL
MEDIA_CREATIVE / ORGANIC_TWEET	Cannot use segments (must be NO_SEGMENT)
CONVERSION_TAGS segment	Not allowed with FUNDING_INSTRUMENT or PROMOTED_ACCOUNT
LANGUAGES segment	Not allowed with ACCOUNT, FUNDING_INSTRUMENT, or MEDIA_CREATIVE

Usage Examples

from mage_integrations.sources.twitter_ads.tap_twitter_ads.schema import get_schemas

reports = [
    {'name': 'campaign_report', 'entity': 'CAMPAIGN', 'segment': 'NO_SEGMENT', 'granularity': 'DAY'},
]
schemas, field_metadata = get_schemas(reports, logger=logger)

Related Pages

Implements Principle

• Principle:Mage_ai_Mage_ai_JSON_Schema_Loading

Requires Environment

• Environment:Mage_ai_Mage_ai_Python_3_9_Runtime