Implementation:Mage ai Mage ai Twitter Ads Sync

Knowledge Sources	Mage Integrations Twitter Ads API
Domains	Data_Integration, Twitter_Ads, Sync
Last Updated	2026-02-09 00:00 GMT

Overview

Sync orchestration module for the Mage Twitter Ads source connector, responsible for coordinating the extraction of data across selected streams, report streams, and multiple Twitter Ads accounts.

Description

The sync function orchestrates the full data extraction lifecycle for the Twitter Ads tap. It implements a three-level nested loop structure:

1. Account loop - Iterates through each account_id from the comma-separated account_ids config.

2. Parent stream loop - For each account, syncs all selected parent streams (streams without a parent_stream). For each stream:

  - Updates currently_syncing state for resumption
  - Writes Singer schema
  - Logs selected fields
  - Calls stream_obj.sync_endpoint() with full context (client, catalog, state, config, account_id, child_streams, selected_streams)
  - Ensures dependent parent streams are included even if not explicitly selected (when their children are selected)

3. Report stream loop - If any report streams are selected, first fetches targeting values:

  - Country targeting IDs via targeting_criteria/locations for configured country codes
  - Platform targeting IDs via targeting_criteria/platforms
  Then syncs each selected report via reports_obj.sync_report() with country_ids and platform_ids.

The function maintains parent-child stream relationships, ensuring parents are synced before children even when parents are not explicitly selected.

Usage

Called by the tap entry point after discovery and catalog selection to execute the data extraction.

Code Reference

Source Location

• Repository: mage-ai
• File: mage_integrations/mage_integrations/sources/twitter_ads/tap_twitter_ads/sync.py
• Lines: 1-177

Signature

def sync(client, config, catalog, state, logger=LOGGER):

Import

from mage_integrations.sources.twitter_ads.tap_twitter_ads.sync import sync

I/O Contract

Inputs

Name	Type	Required	Description
client	twitter_ads.Client	Yes	Authenticated Twitter Ads SDK client
config	dict	Yes	Configuration with account_ids, country_codes, start_date, reports
catalog	Catalog	Yes	Singer catalog with selected streams
state	dict	Yes	Singer state for bookmark and resumption tracking
logger	Logger	No	Optional logger (defaults to Singer LOGGER)

Outputs

Name	Type	Description
(side effects)	-	Writes Singer SCHEMA, RECORD, and STATE messages to stdout

Sync Flow

Phase	Description
1. Stream classification	Separates streams into parent_streams, child_streams, and report_streams
2. Resumption check	Reads last_stream from state for interrupted sync recovery
3. Account iteration	Loops through each account_id from config
4. Parent stream sync	For each parent stream: write schema, sync endpoint, log totals
5. Targeting value fetch	Fetches country_ids and platform_ids for report context
6. Report stream sync	For each report: write schema, sync report with targeting context

Usage Examples

from mage_integrations.sources.twitter_ads.tap_twitter_ads.sync import sync
from mage_integrations.sources.twitter_ads.tap_twitter_ads import build_client

client = build_client(config)
sync(client=client, config=config, catalog=catalog, state=state)

Related Pages

Implements Principle

• Principle:Mage_ai_Mage_ai_Source_Lifecycle_Orchestration

Requires Environment

• Environment:Mage_ai_Mage_ai_Python_3_9_Runtime