Implementation:Dotnet Machinelearning BinaryClassification Evaluate

Knowledge Sources	ML.NET ML.NET API Reference
Domains	Machine Learning, Model Evaluation, Statistics
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for evaluating binary classification model quality using metrics derived from the confusion matrix.

Description

The BinaryClassificationCatalog.Evaluate method computes calibrated binary classification metrics by comparing predicted labels and probabilities against ground-truth labels on a held-out test set. It produces a CalibratedBinaryClassificationMetrics object containing Accuracy, AUC, F1Score, PositivePrecision (Precision), PositiveRecall (Recall), LogLoss, and LogLossReduction.

CrossValidate partitions the dataset into k folds (default 5), trains a model on k-1 folds, evaluates on the remaining fold, and repeats for all folds. The result is a list of per-fold metrics, enabling robust estimation of model performance with variance.

Both methods operate on the output of model.Transform(testData), which adds prediction columns (PredictedLabel, Score, Probability) to the test IDataView.

Usage

Call Evaluate after transforming the test set with the trained model to get a single-point performance estimate. Use CrossValidate for more robust estimates on smaller datasets or for final model selection. Report AUC as the primary ranking metric, F1 for balanced error cost, and LogLoss for probability calibration quality.

Code Reference

Source Location

Repository: ML.NET
File: src/Microsoft.ML.Data/TrainCatalog.cs:L202-213 (Evaluate)
File: src/Microsoft.ML.Data/TrainCatalog.cs:L270+ (CrossValidate)

Signature

public CalibratedBinaryClassificationMetrics Evaluate(
    IDataView data,
    string labelColumnName = "Label",
    string scoreColumnName = "Score",
    string probabilityColumnName = "Probability",
    string predictedLabelColumnName = "PredictedLabel")

public IReadOnlyList<CrossValidationResult<CalibratedBinaryClassificationMetrics>> CrossValidate(
    IDataView data,
    IEstimator<ITransformer> estimator,
    int numberOfFolds = 5,
    string labelColumnName = "Label",
    string samplingKeyColumnName = null,
    int? seed = null)

Import

using Microsoft.ML;

I/O Contract

Inputs

Name	Type	Required	Description
data	IDataView	Yes	Transformed test data containing prediction columns (PredictedLabel, Score, Probability).
labelColumnName	string	No	Name of the true label column. Default: "Label".
scoreColumnName	string	No	Name of the score column. Default: "Score".
probabilityColumnName	string	No	Name of the probability column. Default: "Probability".
numberOfFolds	int	No	Number of cross-validation folds (CrossValidate only). Default: 5.

Outputs

Name	Type	Description
(return)	CalibratedBinaryClassificationMetrics	Evaluation metrics including Accuracy, AreaUnderRocCurve, F1Score, PositivePrecision, PositiveRecall, LogLoss, LogLossReduction.

Key Metric Properties:

Property	Type	Description
Accuracy	double	Fraction of correct predictions.
AreaUnderRocCurve	double	AUC: probability that a random positive ranks higher than a random negative.
F1Score	double	Harmonic mean of precision and recall.
PositivePrecision	double	Precision: TP / (TP + FP).
PositiveRecall	double	Recall: TP / (TP + FN).
LogLoss	double	Binary cross-entropy of probabilistic predictions. Lower is better.
LogLossReduction	double	Relative reduction vs. random baseline (log(2)). Higher is better.

Usage Examples

Basic Example

using Microsoft.ML;

var mlContext = new MLContext(seed: 42);

// Assume model is a trained ITransformer and testData is an IDataView
var predictions = model.Transform(testData);

var metrics = mlContext.BinaryClassification.Evaluate(predictions);

Console.WriteLine($"Accuracy: {metrics.Accuracy:P2}");
Console.WriteLine($"AUC:      {metrics.AreaUnderRocCurve:F4}");
Console.WriteLine($"F1 Score: {metrics.F1Score:F4}");
Console.WriteLine($"Log-Loss: {metrics.LogLoss:F4}");

Cross-Validation Example

var cvResults = mlContext.BinaryClassification.CrossValidate(
    data: fullData,
    estimator: pipeline,
    numberOfFolds: 5);

var avgAuc = cvResults.Average(r => r.Metrics.AreaUnderRocCurve);
Console.WriteLine($"Average AUC across 5 folds: {avgAuc:F4}");

Related Pages

Implements Principle

Principle:Dotnet_Machinelearning_Binary_Model_Evaluation

Requires Environment

Environment:Dotnet_Machinelearning_Dotnet_SDK_And_Runtime

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment