Environment:Promptfoo Promptfoo Browser Automation
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Browser |
| Last Updated | 2026-02-14 08:00 GMT |
Overview
Optional Playwright-based browser automation environment required for the Browser provider and OpenAI ChatKit provider.
Description
Promptfoo includes a Browser provider for automated web interaction testing and an OpenAI ChatKit provider for web component evaluation. Both require Playwright with Chromium for headless browser automation. These are optional dependencies listed in `optionalDependencies` and are only needed when using browser-based providers. The browser provider supports session persistence for multi-turn conversations, Chrome DevTools Protocol (CDP) connections, and stealth mode via `puppeteer-extra-plugin-stealth`.
Usage
Use this environment when testing web applications via the Browser provider (`provider: browser`) or evaluating OpenAI ChatKit web components. Not required for standard LLM API-based evaluations.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Linux, macOS, Windows | Chromium requires a graphical environment or XVFB on headless Linux |
| Hardware | 2GB+ RAM | For Chromium browser process |
| Display | Optional | Headless mode by default; set `headless: false` for visual debugging |
Dependencies
Node.js Packages (Optional)
- `playwright` >= 1.58.2
- `playwright-extra` >= 4.3.6
- `puppeteer-extra-plugin-stealth` >= 2.11.2
- `@playwright/browser-chromium` >= 1.58.2
System Packages
- Chromium browser (installed via Playwright)
Credentials
- `CDP_DOMAIN`: Chrome DevTools Protocol domain for connecting to existing browser instances (optional)
Quick Install
# Install Playwright and Chromium
npm install playwright
npx playwright install chromium
# Or install all browsers
npx playwright install
Code Evidence
Browser provider is loaded via dynamic import as an optional dependency. The Playwright installation is detected at runtime.
Configuration supports multiple connection modes:
# Example browser provider config
providers:
- id: browser
config:
url: "https://example.com"
headless: true
timeout: 30000
# Optional: connect to existing Chrome instance
# cdpUrl: "ws://localhost:9222"
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `Cannot find module 'playwright'` | Playwright not installed | `npm install playwright` |
| `Browser closed unexpectedly` | Chromium not installed | `npx playwright install chromium` |
| `Error: Chromium revision is not downloaded` | Missing browser binaries | `npx playwright install chromium` |
| `DISPLAY not set` | Headless Linux without XVFB | Use `headless: true` or install xvfb |
Compatibility Notes
- Headless Linux: Chromium runs in headless mode by default. For headed mode on Linux servers, install `xvfb`.
- Docker: Use Playwright's official Docker image or install system dependencies manually.
- Stealth Mode: Uses `puppeteer-extra-plugin-stealth` to avoid bot detection on target websites.
- Session Persistence: Browser state (cookies, localStorage) persists across multi-turn conversations within a session.