Implementation:DevExpress Testcafe CLI Run Orchestration
| Knowledge Sources | |
|---|---|
| Domains | Testing, Web_Automation |
| Last Updated | 2026-02-12 04:00 GMT |
Overview
Concrete test execution orchestration implementation for TestCafe that coordinates argument parsing, test compilation, browser launching, test execution, and result reporting through a fluent API.
Description
The TestCafe CLI orchestration spans two primary files: src/cli/cli.js (L68-191) handles CLI entry and setup, while src/runner/index.js (L800-804) implements the Runner.run() method that executes tests.
The CLI orchestration flow begins with CliArgumentParser.parse() extracting options from process.argv, followed by createTestCafe() initializing the framework with hostname, ports, SSL configuration, proxy settings, and development mode flags. The returned testCafe instance exposes createRunner() which instantiates a Runner with fluent configuration methods: src() for test file sources, browsers() for browser targets, reporter() for output formatting, concurrency() for parallel execution, screenshots() and video() for visual artifacts.
The runner.run(options) method (L800-804) returns a Promise<number> representing the failed test count (0 = all passed). Key run options include speed (0.01-1.0 for action throttling), selectorTimeout (milliseconds to wait for selectors), assertionTimeout (milliseconds for assertion retries), pageLoadTimeout (navigation timeout), stopOnFirstFail (halt on first failure), quarantineMode ({ attemptLimit, successThreshold } for flaky test isolation), debugMode (pause on test start), and debugOnFail (pause on assertion failure).
Internally, Runner delegates to Bootstrapper.createRunnableConfiguration() which creates a Task instance ({ tests, browserConnectionGroups, proxy, opts, runnerWarningLog, messageBus }). The task coordinates test distribution across browsers, event emission to reporters, and result aggregation.
Exit handlers ensure graceful shutdown via async-exit-hook, closing browsers and servers even on SIGINT/SIGTERM.
Usage
Use the CLI orchestration flow when executing TestCafe tests from the command line or programmatically via the Runner API.
Code Reference
Source Location
- Repository: testcafe
- File: src/cli/cli.js (main IIFE), src/runner/index.js (Runner.run)
- Lines: L68-191 (CLI orchestration), L800-804 (Runner.run)
Signature
// CLI entry point
async function runTests(argParser): Promise<number>
// Runner class
class Runner {
src(...sources: string[]): Runner
browsers(...browsers: string[]): Runner
reporter(name: string, outStream?: Writable): Runner
concurrency(n: number): Runner
screenshots(options: ScreenshotOptions): Runner
video(basePath: string, options?: VideoOptions): Runner
run(options?: RunnerRunOptions): Promise<number>
}
// Run options interface
interface RunnerRunOptions {
skipJsErrors?: boolean
quarantineMode?: { attemptLimit: number, successThreshold: number }
debugMode?: boolean
debugOnFail?: boolean
selectorTimeout?: number
assertionTimeout?: number
pageLoadTimeout?: number
speed?: number
stopOnFirstFail?: boolean
disablePageCaching?: boolean
disableScreenshots?: boolean
}
Import
// CLI usage (via bin/testcafe.js)
// testcafe chrome tests/ --reporter spec
// Programmatic usage
import createTestCafe from 'testcafe';
const testcafe = await createTestCafe('localhost', 1337, 1338);
const runner = testcafe.createRunner();
const failedCount = await runner
.src(['tests/**/*.js'])
.browsers(['chrome', 'firefox'])
.reporter('spec')
.run();
await testcafe.close();
process.exit(failedCount);
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| sources | string[] | Yes | Test file paths or glob patterns |
| browsers | string[] | Yes | Browser aliases ('chrome', 'firefox:headless', etc.) |
| reporter | object | No | Reporter name or configuration (default: 'spec') |
| concurrency | number | No | Number of parallel browser instances (default: 1) |
| speed | number | No | Test execution speed 0.01-1.0 (default: 1) |
| selectorTimeout | number | No | Selector resolution timeout in ms (default: 10000) |
| assertionTimeout | number | No | Assertion retry timeout in ms (default: 3000) |
| pageLoadTimeout | number | No | Page navigation timeout in ms (default: 3000) |
| quarantineMode | object | No | { attemptLimit: number, successThreshold: number } |
| debugMode | boolean | No | Enable debugging mode (default: false) |
| stopOnFirstFail | boolean | No | Stop on first test failure (default: false) |
Outputs
| Name | Type | Description |
|---|---|---|
| Exit code | number | Number of failed tests (0 = all passed, >0 = failure count) |
| Reporter output | side effect | Formatted test results written to configured streams |
| Screenshots | files | Screenshot files if configured |
| Videos | files | Video files if configured |
Usage Examples
Basic CLI Test Execution
// Command line
testcafe chrome tests/ --reporter spec
// Equivalent programmatic API
import createTestCafe from 'testcafe';
async function runBasicTest() {
const testcafe = await createTestCafe('localhost', 1337, 1338);
const runner = testcafe.createRunner();
const failedCount = await runner
.src('tests/')
.browsers('chrome')
.reporter('spec')
.run();
await testcafe.close();
return failedCount;
}
Advanced Configuration
import createTestCafe from 'testcafe';
import fs from 'fs';
async function runAdvancedTest() {
const testcafe = await createTestCafe({
hostname: 'localhost',
port1: 1337,
port2: 1338,
ssl: {
key: fs.readFileSync('key.pem'),
cert: fs.readFileSync('cert.pem')
}
});
const runner = testcafe.createRunner();
const failedCount = await runner
.src(['tests/**/*.js', 'tests/**/*.ts'])
.browsers(['chrome:headless', 'firefox:headless'])
.reporter('spec')
.reporter('json', fs.createWriteStream('report.json'))
.concurrency(3)
.screenshots({
path: './screenshots',
takeOnFails: true,
pathPattern: '${DATE}_${TIME}/${FIXTURE}/${TEST}/${BROWSER}.png'
})
.video('./videos', {
failedOnly: true,
singleFile: false
})
.run({
skipJsErrors: false,
quarantineMode: {
attemptLimit: 5,
successThreshold: 3
},
selectorTimeout: 5000,
assertionTimeout: 2000,
pageLoadTimeout: 8000,
speed: 0.5,
stopOnFirstFail: false
});
await testcafe.close();
return failedCount;
}
Debugging Configuration
// Enable debug mode to pause on test start
const failedCount = await runner
.src('tests/problematic-test.js')
.browsers('chrome')
.run({
debugMode: true, // Pause on test start
debugOnFail: true, // Pause on assertion failure
speed: 0.1 // Slow down actions
});
// Use with Chrome DevTools:
// 1. Run test with debugMode
// 2. Open Chrome DevTools in test browser
// 3. Set breakpoints in test code
// 4. Resume execution from TestCafe debug panel
Quarantine Mode for Flaky Tests
// Quarantine mode re-runs failed tests to detect flakiness
const failedCount = await runner
.src('tests/flaky-tests/')
.browsers('chrome')
.run({
quarantineMode: {
attemptLimit: 5, // Run up to 5 times
successThreshold: 3 // Pass if 3+ attempts succeed
}
});
// Test marked as passed if it passes 3 out of 5 attempts
// Useful for tests with timing issues or external dependencies
Multiple Browsers with Concurrency
// Run tests across multiple browsers in parallel
const failedCount = await runner
.src('tests/')
.browsers([
'chrome',
'firefox',
'safari',
'edge'
])
.concurrency(4) // Run 4 browser instances in parallel
.run();
// Each browser runs full test suite
// Concurrency splits tests across browser instances
// Total execution time ≈ (tests / concurrency) * avg_test_time
Stop on First Fail
// Useful for fail-fast in development
const failedCount = await runner
.src('tests/')
.browsers('chrome')
.run({
stopOnFirstFail: true // Halt execution on first failure
});
// Saves time when debugging
// Prevents cascading failures
CLI Orchestration Flow
// Internal CLI flow from src/cli/cli.js
async function runTests(argParser) {
const opts = argParser.opts;
// 1. Create TestCafe instance
const testCafe = await createTestCafe({
hostname: opts.hostname,
port1: opts.ports?.[0],
port2: opts.ports?.[1],
ssl: opts.ssl,
developmentMode: opts.dev
});
try {
// 2. Create and configure runner
const runner = testCafe.createRunner();
runner
.useProxy(opts.proxy, opts.proxyBypass)
.src(opts.src)
.browsers(opts.browsers)
.concurrency(opts.concurrency)
.filter(opts.testGrep, opts.fixtureGrep, opts.testMeta, opts.fixtureMeta)
.tsConfigPath(opts.tsConfigPath);
// 3. Configure reporters
opts.reporters.forEach(r => {
runner.reporter(r.name, r.outStream);
});
// 4. Configure screenshots/video
if (opts.screenshots)
runner.screenshots(opts.screenshots);
if (opts.videoPath)
runner.video(opts.videoPath, opts.videoOptions);
// 5. Run tests
const failedCount = await runner.run(opts);
// 6. Return exit code
return failedCount;
}
finally {
// 7. Cleanup
await testCafe.close();
}
}
Exit Code Handling
// Exit codes indicate test results
async function main() {
const testcafe = await createTestCafe();
const runner = testcafe.createRunner();
const failedCount = await runner
.src('tests/')
.browsers('chrome')
.run();
await testcafe.close();
// Exit codes:
// 0 = all tests passed
// 1+ = number of failed tests
// Non-zero exit code fails CI pipelines
process.exit(failedCount);
}
// CI integration
// if [ $? -eq 0 ]; then
// echo "Tests passed"
// else
// echo "Tests failed"
// exit 1
// fi