Environment:Astronomer Astronomer cosmos Cosmos Airflow Configuration

Knowledge Sources	astronomer-cosmos Cosmos Configuration
Domains	Infrastructure, Configuration
Last Updated	2026-02-07 17:00 GMT

Overview

Airflow configuration section [cosmos] with 20+ settings controlling caching, execution, telemetry, and debug behavior.

Description

Cosmos reads its runtime configuration from the Airflow [cosmos] configuration section (typically airflow.cfg or environment variables in the format AIRFLOW__COSMOS__SETTING_NAME). This configuration defines cache directories, feature flags for multi-layer caching, dataset handling, virtual environment management, remote storage paths, telemetry controls, and debug mode settings. All settings have sensible defaults and are optional.

Usage

Configure this environment when you need to customize Cosmos behavior beyond defaults. Key scenarios include: enabling remote caching, disabling telemetry, configuring the Watcher execution queue, or enabling memory-optimised imports for large deployments.

System Requirements

Category	Requirement	Notes
Airflow Config	`[cosmos]` section in airflow.cfg	Or equivalent environment variables
Cache Dir	Writable filesystem path	Defaults to system temp directory
Remote Cache	Object storage access (S3/GCS/Azure)	Only if remote caching is enabled

Dependencies

Python Packages

psutil (optional, required only when enable_debug_mode = True)

Credentials

remote_cache_dir_conn_id: Airflow connection ID for remote cache storage (if remote caching enabled)
remote_target_path_conn_id: Airflow connection ID for remote target path (if configured)
dbt_docs_conn_id: Airflow connection ID for dbt docs hosting (if configured)

Quick Install

# Set via environment variables (no file editing needed)
export AIRFLOW__COSMOS__ENABLE_CACHE=True
export AIRFLOW__COSMOS__CACHE_DIR=/tmp/cosmos
export AIRFLOW__COSMOS__ENABLE_TELEMETRY=True
export AIRFLOW__COSMOS__ENABLE_DEBUG_MODE=False

# Or add to airflow.cfg:
# [cosmos]
# enable_cache = True
# cache_dir = /tmp/cosmos
# enable_telemetry = True
# enable_debug_mode = False

Code Evidence

Full settings module from cosmos/settings.py:17-92:

cache_dir = Path(conf.get("cosmos", "cache_dir", fallback=DEFAULT_CACHE_DIR) or DEFAULT_CACHE_DIR)
enable_cache = conf.getboolean("cosmos", "enable_cache", fallback=True)
enable_dataset_alias = conf.getboolean("cosmos", "enable_dataset_alias", fallback=True)
enable_cache_partial_parse = conf.getboolean("cosmos", "enable_cache_partial_parse", fallback=True)
enable_cache_package_lockfile = conf.getboolean("cosmos", "enable_cache_package_lockfile", fallback=True)
enable_cache_dbt_ls = conf.getboolean("cosmos", "enable_cache_dbt_ls", fallback=True)
enable_cache_dbt_yaml_selectors = conf.getboolean("cosmos", "enable_cache_dbt_yaml_selectors", fallback=True)
enable_cache_profile = conf.getboolean("cosmos", "enable_cache_profile", fallback=True)
virtualenv_max_retries_lock = conf.getint("cosmos", "virtualenv_max_retries_lock", fallback=120)

Memory optimization setting from cosmos/settings.py:48-53:

# Eager imports in cosmos/__init__.py expose all Cosmos classes at the top level,
# which can significantly increase memory usage—even when Cosmos is installed but not actively used.
# This option allows disabling those eager imports to reduce memory footprint.
enable_memory_optimised_imports = conf.getboolean(
    "cosmos", "enable_memory_optimised_imports", fallback=False
)

Debug mode requiring psutil from cosmos/debug.py:18-20:

if not importlib.util.find_spec("psutil"):
    raise RuntimeError(
        "psutil is required for debug mode. Install it with: pip install psutil"
    )

Telemetry opt-out chain from cosmos/settings.py:86-88:

enable_telemetry = conf.getboolean("cosmos", "enable_telemetry", fallback=True)
do_not_track = convert_to_boolean(os.getenv("DO_NOT_TRACK"))
no_analytics = convert_to_boolean(os.getenv("SCARF_NO_ANALYTICS"))

Common Errors

Error Message	Cause	Solution
`RuntimeError: psutil is required for debug mode`	Debug mode enabled without psutil	`pip install psutil` or set `enable_debug_mode = False`
Cache directory permission errors	Temp dir not writable	Set `AIRFLOW__COSMOS__CACHE_DIR` to a writable path
Remote cache connection failure	Connection ID not configured	Set `remote_cache_dir_conn_id` in the `[cosmos]` config section

Compatibility Notes

macOS: The default temp directory path may change between runs; set TMPDIR environment variable for stability
Remote cache: Marked as experimental in Cosmos 1.6; to be merged with cache_dir in future releases
Memory-optimised imports: When enabled, users must use full module paths (e.g., from cosmos.config import ProjectConfig) instead of top-level imports

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment