Principle:Nautechsystems Nautilus trader Backtest Execution

Field	Value
sources	https://github.com/nautechsystems/nautilus_trader , https://nautilustrader.io/docs/
domains	backtesting, event-driven simulation, execution
last_updated	2026-02-10 12:00 GMT

Overview

Backtest Execution is the principle of replaying historical market data through an event-driven simulation loop that advances a simulated clock, dispatches data to matching engines, processes time events, and drives strategy callbacks in exact chronological order.

Description

The execution phase is the heart of backtesting. After the engine has been configured with venues, instruments, data, and strategies, the run() method initiates the main simulation loop. This loop iterates over every data event in timestamp order, advancing the simulated clock to each event's time, feeding the event to the appropriate simulated exchange for order matching, dispatching it through the data engine for strategy consumption, and processing any time-based events (scheduled callbacks, timer alerts) that fall within the gap between consecutive data timestamps.

The principle addresses the following concerns:

• Temporal fidelity -- The simulated clock advances discretely from one data event to the next. There is no concept of "wall time" -- the engine can process years of data in seconds. All components see a consistent, monotonically increasing time.
• Event ordering -- Data events are processed in ts_init order. Within the same timestamp, events are processed in insertion order. Timer events that fire between two data timestamps are processed during the clock advance step.
• Exchange-data coupling -- Each data event is routed to its venue's simulated exchange before being dispatched to the data engine. This means the matching engine sees new prices and can trigger fills before the strategy's on_data callback fires, mimicking the real-world sequence where the exchange processes a trade before notifying participants.
• Streaming mode -- For datasets larger than available memory, the engine supports a streaming workflow: load a batch, run with streaming=True, clear data, load the next batch, and continue. The engine maintains state (positions, orders, account balances) across batches.
• Error containment -- If an AccountError occurs (e.g., insufficient margin), the engine sets a FORCE_STOP flag and halts gracefully rather than corrupting state. In streaming mode, the exception is re-raised to interrupt batch processing.
• Post-run finalization -- After the data loop completes, the engine stops all sub-engines (data, execution, risk, emulator), processes remaining exchange messages, flushes timer events up to the end time, and logs post-run statistics.

Usage

Apply this principle whenever you need to:

• Execute a backtest after all setup steps are complete.
• Understand the event ordering semantics of the simulation (data before strategy, exchange before data engine).
• Implement streaming backtests for large datasets.
• Debug timing-related issues in strategy callbacks.

Theoretical Basis

Event-driven backtest execution is modeled as a discrete-event simulation (DES) where the simulation clock advances to the timestamp of the next event, rather than ticking at a fixed interval.

Key theoretical elements:

• Event priority queue -- The data iterator provides events in sorted order. Timer events are accumulated by the Rust-backed TimeEventAccumulator and interleaved with data events during clock advancement. The merged stream forms a virtual priority queue ordered by timestamp.
• Clock advance protocol -- When the next data event has a timestamp greater than the current clock time, the engine calls _advance_time(ts), which: (1) advances all component clocks, (2) collects timer event handlers that fired in the interval, (3) returns raw handler references for deferred processing. Timer handlers are executed after all data at the same timestamp has been processed.
• Matching engine cascade -- For each data event, the engine dispatches it to the appropriate exchange type handler (process_quote_tick, process_trade_tick, process_bar, etc.), then calls exchange.process(ts) to flush execution messages. This two-phase approach separates market state update from order matching.
• Start/end windowing -- The start and end parameters define a time window. Data before start is skipped (the iterator is fast-forwarded), and data after end terminates the loop. If not specified, the engine uses the first and last data timestamps.
• Kernel lifecycle -- The first call to run() triggers the kernel start sequence (initializes accounts, starts sub-engines, calls strategy on_start). Subsequent calls (in streaming mode) skip this initialization.

Pseudocode:

FUNCTION run(engine, start, end, run_config_id, streaming):
    VALIDATE data is sorted
    RESOLVE start, end from data bounds if not specified
    VALIDATE start <= end

    IF first run:
        INITIALIZE accounts for all venues
        START kernel (data engine, exec engine, risk engine, strategies)

    SET data iterator to first event >= start

    WHILE True:
        data = iterator.next()

        IF data is None:
            done = process_pending_timers()
            IF done: BREAK

        IF data.ts_init > end:
            BREAK

        IF data.ts_init > last_timestamp:
            raw_handlers = advance_clock(data.ts_init)
            last_timestamp = data.ts_init

        ROUTE data to simulated exchange by type
        DISPATCH data through data engine
        PROCESS exchange messages

        IF next data has different timestamp:
            EXECUTE deferred timer handlers

        iteration += 1

    IF NOT streaming:
        STOP trader, data engine, exec engine, risk engine
        PROCESS remaining exchange messages
        FLUSH timer events
        LOG post-run statistics

Related Pages

• Implementation:Nautechsystems_Nautilus_trader_BacktestEngine_Run
• Heuristic:Nautechsystems_Nautilus_trader_Streaming_Mode_For_Large_Backtests