Principle:Online ml River Multi Output Learning
| Knowledge Sources | |
|---|---|
| Domains | Online_Learning, Multi_Target_Prediction |
| Last Updated | 2026-02-08 18:00 GMT |
Overview
Multi-output learning addresses prediction tasks where each input instance is associated with multiple target variables simultaneously. This encompasses both multi-label classification (predicting a subset of labels from a fixed label set) and multi-target regression (predicting multiple continuous values). The central challenge is to exploit statistical dependencies among outputs so that joint predictions are more accurate than independent per-output models.
In the online setting, multi-output learners must update their models incrementally while capturing inter-output correlations from a single pass over the data.
Theoretical Basis
Classifier Chains
A classifier chain imposes an ordering on the L outputs and trains L base models sequentially. Model l receives the original feature vector augmented with the predictions of models 1, ..., l-1:
h_l(x) = f_l(x, h_1(x), h_2(x), ..., h_{l-1}(x))
This allows downstream models to condition on upstream predictions, capturing label dependencies without full combinatorial enumeration. Chain order can be random, fixed, or optimized.
Binary Relevance
The simplest baseline trains one independent model per output, ignoring inter-output dependencies entirely. While computationally efficient, it cannot model label correlations.
Multi-Label Encoding
Multi-label problems can be transformed into multiclass problems by encoding each unique label combination as a distinct class. This preserves label correlations but suffers from exponential growth in the number of possible classes (2^L for L labels). Practical encoders limit this by only considering observed combinations.
Challenges in Online Multi-Output Learning
- Label dependency drift: Correlations between outputs may change over time in non-stationary streams.
- Partial labels: In real-world streams, not all output labels may be available at each time step.
- Scalability: The number of outputs can be very large (e.g., tagging systems with thousands of labels).