Principle:Scikit learn Scikit learn Multiclass Multioutput

Knowledge Sources	Scikit_learn Scikit-learn Docs
Domains	Supervised Learning, Classification
Last Updated	2026-02-08 15:00 GMT

Overview

Multiclass and multioutput strategies extend binary classifiers to handle problems with more than two classes or multiple simultaneous prediction targets.

Description

Many real-world problems require predicting more than two classes, predicting multiple labels simultaneously, or producing multiple output values. Multiclass strategies decompose a multi-class problem into multiple binary classification sub-problems, enabling binary-only classifiers to handle multi-class tasks. Multioutput strategies handle problems where each sample has multiple target variables. These meta-strategies solve the problem of adapting the large collection of binary classifiers to richer prediction tasks, providing a modular approach where any binary classifier can be extended. They form the bridge between binary classification algorithms and the diverse prediction tasks encountered in practice.

Usage

Use One-vs-Rest (OvR) for multiclass classification when you want to train one binary classifier per class and the classes are mutually exclusive. OvR is the most common strategy and works well when the number of classes is moderate. Use One-vs-One (OvO) when the base classifier scales poorly with dataset size, since each sub-problem uses a smaller subset of data. Use MultiOutputClassifier when each sample has multiple independent target labels and you want to train a separate classifier for each target. Use MultiOutputRegressor for the analogous regression case. Use ClassifierChain or RegressorChain when the targets are correlated and you want to model these dependencies.

Theoretical Basis

One-vs-Rest (OvR): For $K$ classes, train $K$ binary classifiers, one for each class against all others:

Classifier $k$ : Relabels class $k$ as positive and all other classes as negative.
Prediction: $\hat{y} = \arg \max_{k} f_{k} (x)$ , where $f_{k}$ is the decision function of classifier $k$ .
The approach produces $K$ classifiers, each trained on the full dataset.

One-vs-One (OvO): Train a binary classifier for each pair of classes:

Number of classifiers: $(\binom{K}{2}) = \frac{K (K - 1)}{2}$
Prediction: Each classifier votes for one of its two classes; the class with the most votes wins.
Each classifier is trained on only the samples from the two relevant classes, making individual problems smaller.

MultiOutput: For $T$ target variables, train $T$ independent estimators:

Classifier $t$ predicts target $y^{(t)}$ independently.
Prediction: $\hat{y} = [{\hat{y}}^{(1)}, {\hat{y}}^{(2)}, \dots, {\hat{y}}^{(T)}]$
Assumes targets are independent, which simplifies the problem but ignores correlations.

Classifier Chain: Addresses target dependencies by chaining classifiers:

Order the targets $y^{(1)}, y^{(2)}, \dots, y^{(T)}$ .
Classifier $t$ predicts $y^{(t)}$ using the original features $x$ augmented with the predictions of all previous classifiers:
${\hat{y}}^{(t)} = f_{t} (x, {\hat{y}}^{(1)}, \dots, {\hat{y}}^{(t - 1)})$
This models conditional dependencies between targets.

Multilabel classification is a special case of multioutput where each target is binary (indicating presence or absence of a label). A sample can have zero, one, or multiple active labels simultaneously.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment