Environment:Dagster io Dagster PostgreSQL Storage

Knowledge Sources	dagster-postgres Instance Configuration
Domains	Infrastructure, Storage
Last Updated	2026-02-10 12:00 GMT

Overview

PostgreSQL database environment for production-grade Dagster instance storage of runs, events, and schedules.

Description

This environment provides a PostgreSQL-backed storage layer for Dagster, replacing the default SQLite storage. PostgreSQL is the recommended storage backend for production deployments because it supports concurrent access from multiple processes (webserver, daemon, code servers) and provides durability guarantees. The dagster-postgres package implements run storage, event log storage, and schedule storage on top of PostgreSQL via SQLAlchemy.

Usage

Use this environment for any multi-process or production Dagster deployment. It is required when running the webserver, daemon, and code servers as separate processes (the standard production architecture). It is also required for Kubernetes, Docker Compose, and hybrid cloud deployments.

System Requirements

Category	Requirement	Notes
Database	PostgreSQL 12+	Any recent PostgreSQL version
Network	TCP access to PostgreSQL	Default port 5432
Disk	Varies	Database storage grows with run/event history

Dependencies

Python Packages

dagster-postgres - PostgreSQL storage implementation
psycopg2 or psycopg2-binary - PostgreSQL adapter (transitive dependency)

System Packages

libpq-dev (Debian/Ubuntu) or postgresql-devel (RHEL) - Required to compile psycopg2 from source

Credentials

The following credentials are configured in dagster.yaml:

username: PostgreSQL username
password: PostgreSQL password (can use $ENV{"DAGSTER_PG_PASSWORD"} syntax)
hostname: Database server hostname
db_name: Database name
port: Connection port (default: 5432)

Kubernetes-specific:

DAGSTER_PG_PASSWORD: Environment variable injected from Kubernetes Secrets (secret key: postgresql-password)

Quick Install

# Install dagster-postgres
pip install dagster-postgres

# Configure in dagster.yaml
cat > $DAGSTER_HOME/dagster.yaml << 'EOF'
storage:
  postgres:
    postgres_db:
      username: dagster
      password:
        env: DAGSTER_PG_PASSWORD
      hostname: localhost
      db_name: dagster
      port: 5432
EOF

Code Evidence

PostgreSQL connection configuration from storage/config.py:35-66:

# PostgreSQL configuration schema
"postgres_db": {
    "username": str,
    "password": str,
    "hostname": str,
    "db_name": str,
    "port": int,  # default: 5432
    "params": dict,  # optional
    "scheme": str,  # default: "postgresql"
}

Kubernetes secret injection from dagster-k8s/job.py:41-47:

DAGSTER_PG_PASSWORD_ENV_VAR = "DAGSTER_PG_PASSWORD"
DAGSTER_PG_PASSWORD_SECRET_KEY = "postgresql-password"

Common Errors

Error Message	Cause	Solution
`could not connect to server: Connection refused`	PostgreSQL not running or wrong host/port	Verify PostgreSQL is running and accessible
`FATAL: password authentication failed`	Wrong credentials	Check username/password in dagster.yaml
`No module named 'dagster_postgres'`	Package not installed	`pip install dagster-postgres`
`error: pg_config not found`	libpq-dev missing (compiling psycopg2)	`apt install libpq-dev` or use `psycopg2-binary`

Compatibility Notes

Auto-creates tables: By default, Dagster auto-creates storage tables via Alembic migrations (should_autocreate_tables: true).
Alternative URL format: Can provide a full connection string via postgres_url instead of individual fields.
MySQL alternative: dagster-mysql provides MySQL storage with similar configuration (default port 3306).
SQLite limitation: SQLite does not support concurrent writes, making it unsuitable for multi-process production deployments.

Related Pages

Note: PostgreSQL storage is an optional deployment configuration, not a strict requirement for any implementation. The implementations below benefit from PostgreSQL in production deployments.

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment