Principle:Dotnet Machinelearning Experiment Execution

Knowledge Sources	ML.NET ML.NET Docs
Domains	Machine_Learning, AutoML
Last Updated	2026-02-09 00:00 GMT

Overview

AutoML experiment execution orchestrates the iterative search loop where a tuner proposes hyperparameter configurations, trials train and evaluate models, and the process continues until the time budget expires or cancellation is requested.

Description

The experiment execution phase is where the AutoML engine performs the actual model search. Once the experiment is fully configured (pipeline, data, metric, budget), calling Run or RunAsync initiates the following loop:

1. The tuner proposes a new set of hyperparameters (including the algorithm choice) from the search space.
2. A trial is created: the sweepable pipeline is materialized into a concrete ML pipeline using the proposed hyperparameters.
3. The concrete pipeline is trained on the training data.
4. The resulting model is evaluated on the validation data using the configured metric.
5. The trial result (metric value, training duration, resource usage) is reported back to the tuner.
6. The tuner updates its model of the search space based on the new observation.
7. The loop repeats from step 1 until the time budget is exhausted or a cancellation token is triggered.

Throughout execution, the engine tracks the best trial seen so far. Each new trial result is compared against the current best, and the overall best is updated if the new result is superior.

The execution supports both synchronous and asynchronous modes. The synchronous Run() method blocks the calling thread until completion. The asynchronous RunAsync() method returns a Task and accepts a CancellationToken, enabling integration with async workflows and graceful early termination.

Usage

Call Run() for simple scripts and console applications where blocking is acceptable. Use RunAsync() in web applications, UI applications, or any context where you need non-blocking execution or the ability to cancel the search programmatically. Always handle the possibility that the experiment may complete with suboptimal results if the time budget is too short for the complexity of the search space.

Theoretical Basis

The AutoML execution loop implements a Sequential Model-Based Optimization (SMBO) pattern:

best_result = null
while time_remaining > 0 and not cancelled:
    config = tuner.propose(history)
    pipeline = materialize(sweepable_pipeline, config)
    model = pipeline.fit(D_train)
    metric = evaluate(model, D_validation)
    loss = to_loss(metric)
    history.append((config, loss))
    tuner.update(config, loss)
    if best_result is null or loss < best_result.loss:
        best_result = (model, metric, loss, config)
return best_result

When the tuner is Bayesian (the default), it maintains a surrogate model (typically a tree-structured Parzen estimator or Gaussian process) that approximates the objective function across the search space. Each new observation refines the surrogate, focusing subsequent proposals on regions of the search space most likely to yield improvements. This is more sample-efficient than random or grid search, particularly in high-dimensional hyperparameter spaces.

The CancellationToken mechanism enables cooperative cancellation: the execution loop checks the token between trials and exits cleanly if cancellation is requested, ensuring the best result found so far is returned rather than lost.

Related Pages

Implemented By

• Implementation:Dotnet_Machinelearning_AutoMLExperiment_Run

Uses Heuristic

• Heuristic:Dotnet_Machinelearning_AutoML_SMAC_Dimension_Limit