Jump to content

Connect Leeroopedia MCP: Equip your AI agents to search best practices, build plans, verify code, diagnose failures, and look up hyperparameter defaults.

Environment:Pola rs Polars Python Runtime Environment

From Leeroopedia


Knowledge Sources
Domains Infrastructure, Data_Engineering
Last Updated 2026-02-09 10:00 GMT

Overview

Python 3.10+ runtime environment for the Polars DataFrame library (v1.38.1), with optional dependencies for interop, cloud storage, databases, Excel, and GPU acceleration.

Description

This environment defines the Python runtime requirements for using the Polars library. The core installation has minimal dependencies (only the Polars native runtime). Optional feature flags enable interoperability with NumPy, Pandas, PyArrow, cloud storage providers, database engines, Excel formats, and NVIDIA GPU acceleration. Polars uses PyO3 to bind to a Rust-compiled native extension, so no Rust toolchain is needed for end users.

Usage

Use this environment for any Python workflow that uses the Polars library, including all documented workflows: Lazy Query Pipeline, Data I/O, DataFrame Aggregation, Time Series Analysis, SQL Query Interface, and Streaming Large Dataset Processing.

System Requirements

Category Requirement Notes
OS Linux, macOS, Windows All major platforms supported
Python >= 3.10 3.10, 3.11, 3.12, 3.13 officially tested
CPU x86_64 with AVX2 (default) Use `polars[rtcompat]` for legacy CPUs without AVX
Disk Minimal Only needed for temporary files during streaming

Dependencies

Core Package

  • `polars` >= 1.38.1 (includes `polars-runtime-32` native extension)

Optional Python Packages

Interoperability:

  • `numpy` >= 1.16.0
  • `pandas` (requires pyarrow)
  • `pyarrow` >= 7.0.0
  • `pydantic`

Excel:

  • `fastexcel` >= 0.9 (calamine engine)
  • `openpyxl` >= 3.0.0
  • `xlsx2csv` >= 0.8.0
  • `xlsxwriter`

Database:

  • `adbc-driver-manager[dbapi]`
  • `connectorx` >= 0.3.2
  • `sqlalchemy`

Cloud:

  • `fsspec`
  • `polars_cloud` >= 0.4.0

Other I/O:

  • `deltalake` >= 1.0.0
  • `pyiceberg` >= 0.7.1

GPU:

Other:

  • `gevent` (async collect)
  • `cloudpickle` (UDF serialization)
  • `matplotlib` (query plan visualization)
  • `altair` >= 5.4.0 (plot namespace)
  • `great-tables` >= 0.8.0 (style namespace)
  • `tzdata` (timezone support on Windows)

Runtime Variants

  • `polars[rt64]` - 64-bit index runtime, supports > 4.3 billion rows (uses `polars-runtime-64`)
  • `polars[rtcompat]` - Legacy CPU compatibility runtime (no AVX2 required)

Credentials

No credentials required for core functionality. Cloud storage operations require credentials configured through the Environment:Pola_rs_Polars_Cloud_Storage_Environment.

Quick Install

# Core install
pip install polars

# Install with common optional dependencies
pip install 'polars[numpy,pandas,pyarrow]'

# Install everything
pip install 'polars[all]'

# For legacy CPUs without AVX2
pip install 'polars[rtcompat]'

# For datasets with > 4.3 billion rows
pip install 'polars[rt64]'

Code Evidence

Python version requirement from `py-polars/pyproject.toml:15`: 

requires-python = ">=3.10"

Runtime dependency from `py-polars/pyproject.toml:36`: 

dependencies = ["polars-runtime-32 == 1.38.1"]

Optional dependency definitions from `py-polars/pyproject.toml:44-92`: 

[project.optional-dependencies]
numpy = ["numpy >= 1.16.0"]
pandas = ["pandas", "polars[pyarrow]"]
pyarrow = ["pyarrow >= 7.0.0"]
gpu = ["cudf-polars-cu12"]
rt64 = ["polars-runtime-64 == 1.38.1"]
rtcompat = ["polars-runtime-compat == 1.38.1"]

Common Errors

Error Message Cause Solution
`Illegal instruction (core dumped)` CPU lacks AVX2 support Install with `pip install polars[rtcompat]`
`ImportError: No module named 'pyarrow'` PyArrow not installed for pandas interop `pip install polars[pyarrow]`
`ImportError: No module named 'xlsxwriter'` Excel write dependency missing `pip install polars[xlsxwriter]`
`ImportError: No module named 'connectorx'` Database connector not installed `pip install polars[database]`
`ModuleNotFoundError: No module named 'fsspec'` Cloud filesystem not installed `pip install polars[fsspec]`

Compatibility Notes

  • Windows: Requires `tzdata` package for timezone support (included in `polars[timezone]`)
  • Legacy CPUs: Default binary requires AVX2 instruction set; use `polars[rtcompat]` for older CPUs
  • Big Index: Default runtime supports up to ~4.3 billion rows; use `polars[rt64]` for larger datasets at a small performance cost due to reduced cache efficiency
  • Python 3.14: Supported and tested in CI as of Polars 1.38.1

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment
Retrieved from "https://leeroopedia.com/index.php?title=Environment:Pola_rs_Polars_Python_Runtime_Environment&oldid=27302"