Environment:Lakeraai Pint benchmark Python 310 With Pandas

Knowledge Sources	PINT Benchmark Pandas Documentation
Domains	Data_Engineering, Benchmarking, Infrastructure
Last Updated	2026-02-14 15:00 GMT

Overview

Python 3.10+ Jupyter Notebook environment with pandas, ruamel.yaml, tqdm, requests, and python-dotenv for running the PINT Benchmark evaluation loop and dataset loading.

Description

This environment provides the core runtime context for the PINT Benchmark notebook. It handles dataset loading from YAML files (via ruamel.yaml), DataFrame manipulation and result aggregation (via pandas), progress bars during evaluation (via tqdm), HTTP API calls to external detection services (via requests), and environment variable management (via python-dotenv). The notebook is the primary execution context, managed by jupyter.

Usage

Use this environment for all PINT Benchmark workflows: Hugging Face model evaluation, custom system evaluation, and custom dataset benchmarking. It is the base environment required to run the pint-benchmark.ipynb notebook, load datasets, execute the benchmark loop, and compute scores.

System Requirements

Category	Requirement	Notes
OS	Linux, macOS, or Windows	Standard Python-supported platforms
Hardware	CPU (minimum)	No GPU required for the benchmark runner itself
Disk	1GB+ free space	For Python packages and dataset files
Python	3.10+	Specified in pyproject.toml: `python = "^3.10"`

Dependencies

System Packages

Python 3.10 or higher
poetry (package manager, used for project setup)

Python Packages

jupyter >= 1.0.0 (notebook execution, from pyproject.toml)
pandas (DataFrame operations, result aggregation)
ruamel.yaml < 0.18.0 (YAML dataset loading)
tqdm (progress bars during evaluation)
requests (HTTP calls to detection APIs like Lakera Guard)
python-dotenv (loading .env files for API keys and paths)

Dev Dependencies

pre-commit >= 3.7.0 (git hook management)
nbstripout >= 0.7.1 (notebook output stripping)
ruff >= 0.3.5 (Python linting)

Credentials

The following environment variables must be set in a .env file in the benchmark/ directory:

LAKERA_GUARD_API_KEY: Lakera Guard API key (required for evaluating Lakera Guard; not needed for HuggingFace model evaluation)
DEPLOYMENT_ID: Optional Lakera Dashboard deployment ID for log segmentation
DATASET_PATH: Path to the PINT Benchmark dataset YAML file (defaults to ./data/example-dataset.yaml)

Quick Install

# Install via Poetry (recommended)
poetry install

# Or install dependencies directly via pip
pip install jupyter pandas "ruamel.yaml<0.18.0" tqdm requests python-dotenv

Code Evidence

Dependency installation from benchmark/pint-benchmark.ipynb cell-5:

%pip install --quiet --upgrade requests pandas "ruamel.yaml<0.18.0" tqdm python-dotenv

Import declarations from benchmark/pint-benchmark.ipynb cell-7:

import os
import requests

import tqdm

import pandas as pd

from pathlib import Path
from typing import Callable, Literal

from ruamel.yaml import YAML
from dotenv import load_dotenv

Environment variable loading from benchmark/pint-benchmark.ipynb cell-7:

# load the .env file environment variables
load_dotenv()

# load any DATASET_PATH set in the .env or default to the example dataset
DATASET_PATH = os.getenv("DATASET_PATH", "./data/example-dataset.yaml")

API key usage from benchmark/pint-benchmark.ipynb cell-9:

lakera_session.headers.update(
    {"Authorization": f'Bearer {os.environ.get("LAKERA_GUARD_API_KEY")}'}
)

Poetry configuration from pyproject.toml:

[tool.poetry.dependencies]
python = "^3.10"
jupyter = "^1.0.0"

Common Errors

Error Message	Cause	Solution
`ModuleNotFoundError: No module named 'ruamel'`	ruamel.yaml not installed	`pip install "ruamel.yaml<0.18.0"`
`ModuleNotFoundError: No module named 'dotenv'`	python-dotenv not installed	`pip install python-dotenv`
`FileNotFoundError: ... example-dataset.yaml`	DATASET_PATH points to missing file	Set `DATASET_PATH` in `.env` or use default example dataset
`Error when calling Lakera Guard: 401 Unauthorized`	Invalid or missing API key	Set `LAKERA_GUARD_API_KEY` in `.env` file
`requests.exceptions.ConnectionError`	Network issue or API endpoint unreachable	Check internet connection and API URL

Compatibility Notes

ruamel.yaml version: Pinned to < 0.18.0 in the notebook's pip install command. This upper bound is explicitly specified to avoid breaking changes.
Jupyter: The benchmark is designed to run as a Jupyter Notebook. Running as a standalone Python script requires extracting cell contents and managing imports manually.
Poetry: The project uses Poetry for dependency management. Running poetry install sets up the full environment including dev dependencies.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment