Environment:TobikoData Sqlmesh Python Runtime

Knowledge Sources	TobikoData/sqlmesh
Domains	Python, Data Engineering, SQL
Last Updated	2026-02-07 21:00 GMT

Overview

The core Python runtime environment for SQLMesh, providing data transformation framework capabilities with multi-dialect SQL support.

Description

SQLMesh requires Python 3.9 or higher (but not 3.13+) and uses a comprehensive set of dependencies for SQL parsing, data manipulation, and workflow orchestration. The runtime includes SQLGlot for SQL parsing and transpilation, DuckDB as the default execution engine, Pydantic for configuration validation, and Pandas for data operations. The environment supports parallel model loading through os.fork() and includes rich CLI tooling.

Usage

This environment is required for all SQLMesh operations including model development, plan generation, deployment, testing, and CLI usage. It forms the foundation for Context operations, snapshot management, and state synchronization across all supported database engines.

System Requirements

Category	Requirement	Notes
OS	Linux, macOS, Windows	Requires os.fork() support for parallel loading (Unix-like systems)
Python Version	>= 3.9, < 3.13	Python 3.13+ not yet supported
CPU	Any modern CPU	No GPU requirements
Memory	2GB+ recommended	Depends on project size
importlib-metadata	Required for Python < 3.12	Package discovery

Dependencies

System Packages

Python development headers (for building native extensions)
C compiler (for sqlglot[rs] Rust extensions)

Python Packages

sqlglot[rs]~=28.10.0 - SQL parser and transpiler with Rust extensions
duckdb>=0.10.0,!=0.10.3 - Default SQL execution engine
pydantic>=2.0.0 - Configuration validation
pandas<3.0.0 - Data manipulation
dateparser<=1.2.1 - Date parsing utilities
hyperscript>=0.1.0 - Hypermedia scripting
astor - Python AST manipulation
click - CLI framework
croniter - Cron expression parsing
humanize - Human-readable values
ipywidgets - Jupyter widgets support
jinja2 - Template engine
packaging - Version parsing
python-dotenv - Environment variable management
requests - HTTP client
rich[jupyter] - Terminal formatting
ruamel.yaml - YAML processing
tenacity - Retry library
time-machine - Time mocking for tests
json-stream - Streaming JSON parser

Credentials

Environment variables for runtime configuration:

SQLMESH_HOME - Custom SQLMesh home directory (default: ~/.sqlmesh)
SQLMESH_DEBUG - Enable debug logging mode
SQLMESH_RUNTIME_ENVIRONMENT - Runtime environment identifier
SQLMESH_GATEWAY - Gateway selection for multi-gateway setups
SQLMESH_IGNORE_WARNINGS - Suppress warning messages
SQLMESH_DOTENV_PATH - Custom path to .env file
SQLMESH__DISABLE_ANONYMIZED_ANALYTICS - Disable telemetry collection
MAX_FORK_WORKERS - Control parallel model loading worker count

Quick Install

# Create and activate virtual environment
python -m venv .venv
source ./.venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install SQLMesh with core dependencies
pip install sqlmesh

# Or install from repository for development
git clone https://github.com/TobikoData/sqlmesh.git
cd sqlmesh
make install-dev

Code Evidence

# File: pyproject.toml:8
requires-python = ">= 3.9"

# File: pyproject.toml:14-41
dependencies = [
    "astor",
    "click",
    "croniter",
    "dateparser<=1.2.1",
    "duckdb>=0.10.0,!=0.10.3",
    "humanize",
    "hyperscript>=0.1.0",
    "ipywidgets",
    "jinja2",
    "packaging",
    "pandas<3.0.0",
    "pydantic>=2.0.0",
    "python-dotenv",
    "requests",
    "rich[jupyter]",
    "ruamel.yaml",
    "sqlglot[rs]~=28.10.0",
    "tenacity",
    "time-machine",
    "json-stream",
]

# File: sqlmesh/core/constants.py:40-42
# Disable parallel loading if the platform doesn't support os.fork
# Users can also manually disable it by setting this env var to 0
MAX_FORK_WORKERS = int(os.environ.get("MAX_FORK_WORKERS", "0" if not hasattr(os, "fork") else "8"))

# File: sqlmesh/core/constants.py:11
SQLMESH_PATH = Path(os.environ.get("SQLMESH_HOME", Path.home() / ".sqlmesh")).expanduser()

# File: sqlmesh/utils/__init__.py:220
DEBUG = os.environ.get("SQLMESH_DEBUG", "").lower() in ("1", "true", "t", "yes", "y")

Common Errors

Error Message	Cause	Solution
ImportError: No module named 'sqlglot'	SQLMesh not installed or venv not activated	Run pip install sqlmesh or activate virtual environment
Python version 3.13 not supported	Using unsupported Python version	Use Python 3.9 through 3.12
AttributeError: module 'duckdb' has no attribute '...'	Using duckdb 0.10.3 (known buggy version)	Upgrade to duckdb >= 0.10.4
No module named 'importlib_metadata'	Missing dependency for Python < 3.12	Install importlib-metadata package
MAX_FORK_WORKERS warnings	Platform doesn't support os.fork()	Set MAX_FORK_WORKERS=0 to disable parallel loading

Compatibility Notes

Python 3.13+ support is in progress but not yet available
Windows compatibility requires MAX_FORK_WORKERS=0 due to lack of os.fork()
DuckDB version 0.10.3 has known issues and is explicitly excluded
Pandas 3.0.0+ is not yet supported
importlib-metadata is automatically installed for Python < 3.12
SQLGlot Rust extensions ([rs]) provide significant performance improvements
dateparser 1.2.2+ has compatibility issues, pinned to <= 1.2.1

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment