Implementation:Mage ai Mage ai Google Search Console Streams

Knowledge Sources	Mage Integrations Google Search Console API
Domains	Data_Integration, Google_Search_Console, Stream
Last Updated	2026-02-09 00:00 GMT

Overview

Stream configuration dictionary for the Mage Google Search Console source connector, defining all available data streams with their key properties, replication methods, API paths, and request parameters.

Description

This module defines a STREAMS dictionary that serves as the declarative configuration for all Google Search Console data streams. Rather than defining stream classes, each stream is represented as a configuration dictionary with properties including:

• key_properties - Primary key fields for each stream
• replication_method - Either FULL_TABLE or INCREMENTAL
• path - API endpoint path template (with {} placeholder for site URL)
• data_key - Response key containing the data records
• api_method - HTTP method (GET or POST)
• row_limit - Maximum rows per request (10,000 for performance reports)
• body - Request body with aggregationType and dimensions
• bookmark_type - Type of bookmark field (datetime)
• pagination - Pagination strategy (body-based for POST endpoints)
• sub_types - Search types to iterate (web, image, video)

The module defines 2 full-table streams (sites, sitemaps) and 6 incremental performance report streams with varying dimension combinations.

Usage

Consumed by the Google Search Console sync and discover modules to configure API requests and catalog entries.

Code Reference

Source Location

• Repository: mage-ai
• File: mage_integrations/mage_integrations/sources/google_search_console/streams.py
• Lines: 1-118

Signature

STREAMS = {
    'sites': { ... },
    'sitemaps': { ... },
    'performance_report_custom': { ... },
    'performance_report_date': { ... },
    'performance_report_country': { ... },
    'performance_report_device': { ... },
    'performance_report_page': { ... },
    'performance_report_query': { ... },
}

Import

from mage_integrations.sources.google_search_console.streams import STREAMS

Stream Definitions

Stream	Key Properties	Replication	API Method	Dimensions
sites	site_url	FULL_TABLE	GET	-
sitemaps	site_url, path, last_submitted	FULL_TABLE	GET	-
performance_report_custom	site_url, search_type, date	INCREMENTAL	POST	auto aggregation
performance_report_date	site_url, search_type, date	INCREMENTAL	POST	date
performance_report_country	site_url, search_type, date, country	INCREMENTAL	POST	date, country
performance_report_device	site_url, search_type, date, device	INCREMENTAL	POST	date, device
performance_report_page	site_url, search_type, date, page	INCREMENTAL	POST	date, page
performance_report_query	site_url, search_type, date, query	INCREMENTAL	POST	date, query

Usage Examples

from mage_integrations.sources.google_search_console.streams import STREAMS

stream_config = STREAMS['performance_report_date']
# stream_config = {
#     'key_properties': ['site_url', 'search_type', 'date'],
#     'replication_method': 'INCREMENTAL',
#     'replication_keys': ['date'],
#     'path': 'sites/{}/searchAnalytics/query',
#     'data_key': 'rows',
#     'api_method': 'POST',
#     'row_limit': 10000,
#     'body': {'aggregationType': 'byProperty', 'dimensions': ['date']},
#     ...
# }

Related Pages

Implements Principle

• Principle:Mage_ai_Mage_ai_API_Stream_Discovery

Requires Environment

• Environment:Mage_ai_Mage_ai_Python_3_9_Runtime