Implementation:Microsoft Playwright Test Runner CLI
| Knowledge Sources | |
|---|---|
| Domains | Testing, Browser_Automation |
| Last Updated | 2026-02-11 00:00 GMT |
Overview
Concrete tool for running test suites with parallelism, sharding, and generating structured reports, provided by the Playwright library.
Description
The playwright test CLI command is Playwright's test execution entry point. It is implemented as a commander.js subcommand in packages/playwright/src/program.ts (lines 49-81) and delegates to runAllTestsWithConfig in packages/playwright/src/runner/testRunner.ts (lines 459-489) for actual execution.
The command accepts over 30 CLI options that control every aspect of test execution: which tests to run (filter patterns, project selection, grep), how to run them (workers, retries, timeout, headed/headless), what to capture (trace, video, screenshots), and how to report results (multiple reporter formats). These CLI options override corresponding values in the configuration file.
The test runner supports multiple built-in reporters: list (verbose per-test output), line (single-line progress), dot (minimal), json (machine-readable), junit (CI-compatible XML), html (interactive web report), blob (intermediate format for shard aggregation), github (GitHub Actions annotations), and null (no output). The HTML reporter, implemented at reporters/html.ts (lines 56-186), produces a self-contained web application using HttpServer for local serving and yazl for ZIP packaging.
Usage
Use npx playwright test for running the full test suite in CI. Use filtering options (--grep, --project, test file patterns) to run subsets during development. Use --headed and --debug for local debugging. Use --ui to launch the interactive test runner UI. Use --shard to distribute tests across CI machines.
Code Reference
Source Location
- Repository: playwright
- File (CLI definition):
packages/playwright/src/program.ts(lines 49-81) - File (test execution):
packages/playwright/src/runner/testRunner.ts(lines 459-489) - File (run handler):
packages/playwright/src/program.ts(lines 200-260,runTestsfunction) - File (HTML reporter):
packages/playwright/src/reporters/html.ts(lines 56-186)
Signature
// CLI command registration
program
.command('test [test-filter...]')
.description('run tests with Playwright Test')
.action(async (args: string[], options: CLIOptions) => void);
// Internal execution function
async function runAllTestsWithConfig(
config: FullConfigInternal
): Promise<FullResultStatus>;
// Returns: 'passed' | 'failed' | 'timedout' | 'interrupted'
Import
npx playwright test [options] [test-filter...]
This is a CLI command, not a programmatic API. It is invoked from the command line or from scripts.
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| test-filter | string[] |
No | File name patterns to filter which test files to run (e.g., login matches all files containing "login").
|
| --config | string |
No | Path to the configuration file. Default: playwright.config.ts in the current directory.
|
| --headed | boolean |
No | Run tests in headed browser mode (visible browser window). |
| --grep | string |
No | Only run tests matching this regular expression pattern (matched against test title). |
| --grep-invert | string |
No | Only run tests NOT matching this regular expression pattern. |
| --project | string[] |
No | Only run tests from the specified project(s) defined in the config file. |
| --reporter | string |
No | Reporter to use. Built-in options: list, line, dot, json, junit, html, blob, github, null.
|
| --retries | number |
No | Number of times to retry failed tests. Overrides config value. |
| --shard | string |
No | Shard specification in the format current/total (e.g., 1/3). Distributes tests across shards.
|
| --timeout | number |
No | Maximum timeout for each test in milliseconds. Overrides config value. |
| --workers | string | No | Number of parallel workers. Can be absolute (e.g., 4) or percentage (e.g., 50%).
|
| --trace | string |
No | Trace recording mode: on, off, on-first-retry, retain-on-failure, on-all-retries.
|
| --ui | boolean |
No | Launch the interactive UI mode for running and debugging tests. |
| --debug | boolean |
No | Run tests in debug mode with Playwright Inspector. |
| --update-snapshots | boolean |
No | Update expected screenshot and snapshot baselines. |
| --forbid-only | boolean |
No | Fail if test.only is found (useful for CI to prevent accidentally committed focused tests).
|
| --fully-parallel | boolean |
No | Run all tests in parallel regardless of test.describe.serial grouping.
|
| --last-failed | boolean |
No | Only re-run tests that failed in the previous run. |
| --output | string |
No | Directory for test artifacts (screenshots, traces, videos). |
| --pass-with-no-tests | boolean |
No | Exit with code 0 even if no tests are found (useful for filtered CI runs). |
| --quiet | boolean |
No | Suppress stdio output from tests. |
| --repeat-each | number |
No | Run each test N times (useful for finding flaky tests). |
| --list | boolean |
No | List all tests without running them. |
Outputs
| Name | Type | Description |
|---|---|---|
| Exit code | number |
0 when all tests pass, 1 when any test fails, 130 when execution is interrupted (SIGINT).
|
| Console output | string |
Real-time test results streamed to stdout by the selected reporter. Format depends on reporter type. |
| HTML report | Files on disk | When using the html reporter, a self-contained web application is generated in the playwright-report/ directory. Can be served locally via npx playwright show-report.
|
| JSON report | JSON file |
When using the json reporter, a structured JSON file with complete test results, durations, errors, and metadata.
|
| JUnit report | XML file |
When using the junit reporter, a JUnit-compatible XML file for CI dashboard integration (Jenkins, Azure DevOps, etc.).
|
| Blob report | ZIP file |
When using the blob reporter, an intermediate binary format for merging results across shards via npx playwright merge-reports.
|
| Test artifacts | Files on disk | Screenshots, traces, and videos saved to the output directory based on configuration. |
Usage Examples
Basic Example
# Run all tests
npx playwright test
# Run tests matching a pattern
npx playwright test login
# Run a specific test file
npx playwright test tests/auth.spec.ts
# Run tests with grep filter
npx playwright test --grep "should login"
# Run in headed mode for debugging
npx playwright test --headed
# Run with the Playwright Inspector
npx playwright test --debug
CI Configuration Example
# Run with CI-appropriate settings
npx playwright test \
--reporter=junit,html \
--retries=2 \
--workers=4 \
--forbid-only
# Shard across 3 CI machines
# Machine 1:
npx playwright test --shard=1/3 --reporter=blob
# Machine 2:
npx playwright test --shard=2/3 --reporter=blob
# Machine 3:
npx playwright test --shard=3/3 --reporter=blob
# Merge shard results into a single HTML report
npx playwright merge-reports --reporter=html ./blob-reports
Interactive UI Mode
# Launch the interactive test runner UI
npx playwright test --ui
# Show the HTML report from the last run
npx playwright show-report
Project and Trace Options
# Run only Chromium tests
npx playwright test --project=chromium
# Run with trace recording on first retry
npx playwright test --trace=on-first-retry
# Update screenshot baselines
npx playwright test --update-snapshots
# List all discovered tests without running them
npx playwright test --list