Implementation:DevExpress Testcafe Runner Run

Knowledge Sources	TestCafe TestCafe Docs
Domains	Testing, Web_Automation
Last Updated	2026-02-12 04:00 GMT

Overview

Concrete method for executing the complete test lifecycle including compilation, browser launching, test distribution, and result collection provided by TestCafe Runner.

Description

The run method orchestrates the entire test execution process through multiple subsystems. It begins by resetting runner state, then delegates to _prepareAndRunTask which validates options, merges configuration, initializes reporters, and invokes the Bootstrapper to compile tests and resolve browsers. The Bootstrapper creates a RunnableConfiguration containing compiled tests, browser connections, and loaded client scripts.

With the RunnableConfiguration ready, run creates a Task instance that manages BrowserJobs. Each BrowserJob represents test execution in one browser instance, coordinating TestRunControllers for individual tests. The Task emits events through a MessageBus that reporters subscribe to, enabling real-time progress reporting. The method returns a promise that resolves with the count of failed tests when all browser jobs complete.

Usage

Call run after configuring the runner with sources, browsers, and reporters. The method accepts optional runtime options that override or supplement configuration options. The returned promise resolves with a number indicating failed test count, where 0 means all tests passed. Use this return value to determine exit codes in CI/CD pipelines.

Code Reference

Source Location

• Repository: testcafe
• File: src/runner/index.js (run: L800-804), src/runner/bootstrapper.ts (L105-458), src/runner/task/index.ts (L25-172)
• Lines: Multiple files coordinate execution

Signature

run(options?: RunOptions): Promise<number>

interface RunOptions {
    speed?: number;                    // 0.01-1.0, default 1
    selectorTimeout?: number;          // ms, default 10000
    assertionTimeout?: number;         // ms, default 3000
    pageLoadTimeout?: number;          // ms, default 3000
    stopOnFirstFail?: boolean;         // default false
    quarantineMode?: {
        attemptLimit: number;          // default 5
        successThreshold: number;      // default 1
    };
    debugMode?: boolean;               // default false
    debugOnFail?: boolean;             // default false
    skipJsErrors?: boolean | SkipJsErrorsOptions;
    disablePageCaching?: boolean;      // default false
    disablePageReloads?: boolean;      // default false
    disableMultipleWindows?: boolean;  // default false
}

Import

// run is a method on Runner instances
const createTestCafe = require('testcafe');

const testcafe = await createTestCafe();
const runner = testcafe.createRunner();
const failedCount = await runner.src('tests/*.js').browsers('chrome').run();

I/O Contract

Inputs

Name	Type	Required	Description
speed	number	No	Test execution speed multiplier (0.01-1.0, default: 1)
selectorTimeout	number	No	Maximum wait time for selectors in milliseconds (default: 10000)
assertionTimeout	number	No	Maximum wait time for assertions in milliseconds (default: 3000)
pageLoadTimeout	number	No	Maximum wait time for page loads in milliseconds (default: 3000)
stopOnFirstFail	boolean	No	Stop execution after first test failure (default: false)
quarantineMode	object/boolean	No	Retry flaky tests with attempt limits (default: false)
debugMode	boolean	No	Enable debugging mode with breakpoints (default: false)
debugOnFail	boolean	No	Automatically pause on test failures (default: false)
skipJsErrors	boolean/object	No	Skip JavaScript errors during test execution (default: false)
disablePageCaching	boolean	No	Disable browser page caching (default: false)
disablePageReloads	boolean	No	Disable automatic page reloads (default: false)
disableMultipleWindows	boolean	No	Disable multiple window support (default: false)

Outputs

Name	Type	Description
failedCount	Promise<number>	Promise resolving to count of failed tests (0 if all passed)

Usage Examples

Basic Test Execution

const createTestCafe = require('testcafe');

(async () => {
    const testcafe = await createTestCafe();

    try {
        const runner = testcafe.createRunner();

        const failedCount = await runner
            .src('tests/**/*.js')
            .browsers('chrome')
            .reporter('spec')
            .run();

        console.log('Tests failed:', failedCount);
        process.exit(failedCount > 0 ? 1 : 0);
    }
    finally {
        await testcafe.close();
    }
})();

With Runtime Options

const createTestCafe = require('testcafe');

(async () => {
    const testcafe = await createTestCafe();

    try {
        const runner = testcafe.createRunner();

        const failedCount = await runner
            .src('tests/**/*.js')
            .browsers('chrome:headless')
            .reporter('spec')
            .run({
                speed: 0.5,              // Slow down by 50%
                selectorTimeout: 20000,  // 20 second selector timeout
                assertionTimeout: 5000,  // 5 second assertion timeout
                stopOnFirstFail: true,   // Stop after first failure
                skipJsErrors: true       // Ignore JS errors
            });

        console.log('Failed tests:', failedCount);
    }
    finally {
        await testcafe.close();
    }
})();

Quarantine Mode for Flaky Tests

const createTestCafe = require('testcafe');

(async () => {
    const testcafe = await createTestCafe();

    try {
        const runner = testcafe.createRunner();

        const failedCount = await runner
            .src('tests/**/*.js')
            .browsers('chrome')
            .reporter('spec')
            .run({
                quarantineMode: {
                    attemptLimit: 5,        // Try up to 5 times
                    successThreshold: 2     // Need 2 passes to succeed
                }
            });

        console.log('Failed tests:', failedCount);
    }
    finally {
        await testcafe.close();
    }
})();

Debug Mode

const createTestCafe = require('testcafe');

(async () => {
    const testcafe = await createTestCafe();

    try {
        const runner = testcafe.createRunner();

        const failedCount = await runner
            .src('tests/**/*.js')
            .browsers('chrome')
            .reporter('spec')
            .run({
                debugMode: true,      // Enable debugging
                debugOnFail: true,    // Auto-pause on failures
                speed: 0.1            // Very slow execution for debugging
            });

        console.log('Failed tests:', failedCount);
    }
    finally {
        await testcafe.close();
    }
})();

Conditional Execution with Error Handling

const createTestCafe = require('testcafe');

(async () => {
    const testcafe = await createTestCafe();

    try {
        const runner = testcafe.createRunner();

        const failedCount = await runner
            .src('tests/**/*.js')
            .browsers(['chrome', 'firefox'])
            .reporter('spec')
            .concurrency(2)
            .run();

        if (failedCount > 0) {
            console.error(`${failedCount} test(s) failed`);
            process.exit(1);
        }
        else {
            console.log('All tests passed!');
            process.exit(0);
        }
    }
    catch (error) {
        console.error('Test execution error:', error);
        process.exit(2);
    }
    finally {
        await testcafe.close();
    }
})();

CI/CD Integration

const createTestCafe = require('testcafe');

(async () => {
    const testcafe = await createTestCafe();

    try {
        const runner = testcafe.createRunner();

        const isCI = process.env.CI === 'true';

        const failedCount = await runner
            .src('tests/**/*.js')
            .browsers(isCI ? 'chrome:headless' : 'chrome')
            .reporter(isCI ? ['spec', 'json:report.json'] : 'spec')
            .screenshots({
                path: 'screenshots/',
                takeOnFails: true
            })
            .run({
                stopOnFirstFail: false,
                quarantineMode: isCI ? false : { attemptLimit: 3 },
                skipJsErrors: false
            });

        // Exit with appropriate code for CI
        process.exitCode = failedCount > 0 ? 1 : 0;
    }
    catch (error) {
        console.error('Fatal error:', error);
        process.exitCode = 2;
    }
    finally {
        await testcafe.close();
    }
})();

Implementation Details

Execution Flow

1. _resetBeforeRun(): Clears API call flags and message bus listeners
2. _prepareAndRunTask(options): Prepares execution and returns cancelable promise
3. _prepareOptions(options): Merges runtime options with configuration
4. _validateRunOptions(): Validates all options (screenshots, video, browsers, etc.)
5. _prepareReporters(): Loads reporter plugins and initializes Reporter instances
6. _setBootstrapperOptions(): Transfers configuration to Bootstrapper
7. Bootstrapper.createRunnableConfiguration(): Compiles tests, resolves browsers, loads client scripts
8. _createTask(): Creates Task with tests, browser connections, proxy, and options
9. _getTaskResult(): Waits for task completion, handles errors, collects results
10. Task constructor: Creates BrowserJobs, initializes Screenshots, Videos, FixtureHookController
11. BrowserJob: Distributes tests across browsers, manages TestRunControllers
12. MessageBus events: Emits start, test-run-start, test-run-done, done events
13. Result collection: Counts failures from reporter task info

Key Components

• Runner: Orchestrates execution, validates configuration
• Bootstrapper: Compiles tests, resolves browsers, creates browser connections
• Task: Manages browser jobs, coordinates execution across browsers
• BrowserJob: Executes tests in one browser instance
• TestRunController: Manages individual test lifecycle
• MessageBus: Event communication channel between components
• Reporter: Subscribes to events, generates reports

Event Flow

messageBus.emit('start', task)                    // Task starts
messageBus.emit('test-run-start', testRun)        // Each test starts
messageBus.emit('test-action-done', action)       // Each action completes
messageBus.emit('test-run-done', testRun)         // Each test completes
messageBus.emit('done')                            // All tests complete

Error Handling

The execution pipeline uses Promise.race to handle errors from multiple sources:

const promises = [
    taskDonePromise,           // Normal completion
    browserSetErrorPromise,    // Browser connection errors
    taskErrorPromise,          // Task execution errors
    messageBusErrorPromise,    // Message bus errors
    testedApp.errorPromise     // Tested app errors (if using startApp)
];

await Promise.race(promises);

If any error occurs, the system emits 'unhandled-rejection', waits for reporters to flush, disposes all resources, and throws the error.

Related Pages

Implements Principle

• Principle:DevExpress_Testcafe_Programmatic_Test_Run_Execution

Related Implementations

• Implementation:DevExpress_Testcafe_Runner_Fluent_API
• Implementation:DevExpress_Testcafe_TestCafe_CreateRunner
• Implementation:DevExpress_Testcafe_TestCafe_Close

Requires Environment

• Environment:DevExpress_Testcafe_Node_Runtime
• Environment:DevExpress_Testcafe_Chrome_Browser
• Environment:DevExpress_Testcafe_Firefox_Marionette

Uses Heuristic

• Heuristic:DevExpress_Testcafe_Concurrency_Factor_Limit
• Heuristic:DevExpress_Testcafe_MacOS_Browser_Launch_Serialization
• Heuristic:DevExpress_Testcafe_Window_Resize_Correction
• Heuristic:DevExpress_Testcafe_Browser_Connection_Timeouts