Implementation:Mage ai Mage ai Mode Streams
| Knowledge Sources | |
|---|---|
| Domains | Data_Integration, Mode_Analytics, API |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
Defines all Mode Analytics API stream classes and their sync logic for the Mage Mode source connector.
Description
This module contains the stream class hierarchy for the Mode Analytics source connector. The BaseStream base class provides shared attributes and abstract methods for record retrieval. A key distinction of this connector is that Mode API responses nest data under the _embedded key (set as default_data_key), which is unique among the Mage source connectors. Two intermediate classes set replication behavior: IncrementalStream (INCREMENTAL replication) and FullTableStream (FULL_TABLE replication). Eight concrete stream classes represent Mode resources in a hierarchical parent-child structure: SpaceList, Spaces, ReportList, Reports, QueryList, Queries, ChartList, and Charts. The list streams serve as parents that provide IDs for their detail counterparts (e.g., ReportList is the parent of Reports).
Usage
Imported by the Mode source connector to build stream objects, discover available resources, and execute sync operations against the Mode Analytics API.
Code Reference
Source Location
- Repository: mage-ai
- File: mage_integrations/mage_integrations/sources/mode/streams.py
- Lines: 1-310
Signature
class BaseStream:
tap_stream_id = None
replication_method = None
replication_key = None
key_properties = []
path = None
default_data_key = "_embedded"
data_key = None
parent = None
def __init__(self, client: ModeClient, logger=LOGGER): ...
def get_records(self, bookmark_datetime=None, is_parent=False) -> list: ...
def get_parent_data(self, bookmark_datetime=None) -> list: ...
class IncrementalStream(BaseStream):
replication_method = 'INCREMENTAL'
class FullTableStream(BaseStream):
replication_method = 'FULL_TABLE'
Import
from mage_integrations.sources.mode.streams import STREAMS, BaseStream
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| client | ModeClient | Yes | Authenticated Mode Analytics API client |
| bookmark_datetime | datetime | No | Bookmark datetime for incremental sync |
Outputs
| Name | Type | Description |
|---|---|---|
| records | list[dict] | List of record dictionaries extracted from Mode API responses |
Stream Classes
| Stream Name | Class | Replication | Key Properties | Parent |
|---|---|---|---|---|
| space_list | SpaceList | FULL_TABLE | [id] | None |
| spaces | Spaces | FULL_TABLE | [token] | SpaceList |
| report_list | ReportList | FULL_TABLE | [id] | Spaces |
| reports | Reports | FULL_TABLE | [token] | ReportList |
| query_list | QueryList | FULL_TABLE | [id] | Reports |
| queries | Queries | FULL_TABLE | [token] | QueryList |
| chart_list | ChartList | FULL_TABLE | [id] | Queries |
| charts | Charts | FULL_TABLE | [token] | ChartList |
Key Behaviors
Embedded Data Extraction
Mode API responses wrap result data under the _embedded key rather than at the top level. The default_data_key attribute is set to _embedded so that the sync logic correctly navigates to the results array within each API response.
Hierarchical Stream Dependencies
Streams follow a strict parent-child hierarchy: SpaceList provides space IDs, Spaces provides space tokens for ReportList, and so on down to Charts. Each child stream's get_parent_data method instantiates its parent class and calls get_records with is_parent=True to obtain the IDs needed for constructing child API URLs.
Usage Examples
from mage_integrations.sources.mode.streams import STREAMS
from mage_integrations.sources.mode.client import ModeClient
client = ModeClient(config={"token": "xxx", "password": "yyy", "account": "my_org"})
# Instantiate a stream
space_list_cls = STREAMS["space_list"]
space_list_stream = space_list_cls(client)
# Retrieve records
spaces = space_list_stream.get_records()