Principle:Scikit learn Scikit learn Cross Decomposition

Knowledge Sources	Scikit_learn Scikit-learn Docs
Domains	Supervised Learning, Dimensionality Reduction
Last Updated	2026-02-08 15:00 GMT

Overview

Cross decomposition methods find the fundamental relations between two multivariate datasets by projecting them into shared latent spaces that maximize their covariance or correlation.

Description

Cross decomposition techniques simultaneously decompose two data matrices $X$ and $Y$ to find latent components that capture the maximum covariance between them. Unlike standard regression, which predicts $Y$ from $X$, cross decomposition finds joint latent structures shared by both datasets. These methods solve the problem of modeling relationships between two high-dimensional sets of variables, particularly when the variables within each set are highly correlated (multicollinear). Cross decomposition sits at the intersection of dimensionality reduction and regression, commonly applied in chemometrics, neuroimaging, and genomics.

Usage

Use Partial Least Squares (PLS) Regression when predicting a multivariate response $Y$ from a high-dimensional predictor $X$ and ordinary least squares fails due to multicollinearity or high dimensionality. PLS is the standard method in chemometrics for relating spectral measurements to chemical properties. Use Canonical Correlation Analysis (CCA) when the goal is to find maximally correlated linear combinations of two sets of variables, without a predictor-response distinction. CCA is common in neuroscience for relating brain activity patterns to behavioral measures.

Theoretical Basis

Partial Least Squares (PLS) Regression: PLS finds weight vectors $w_k$ and $c_k$ that maximize the covariance between the latent components:

$(w_k,c_k)=\arg \underset{w,c}{\max }\text{cov}(Xw,Yc)=\arg \underset{w,c}{\max }{w}^T{X}^{T}Yc$

subject to $\Vert w\Vert =1$, $\Vert c\Vert =1$.

The PLS algorithm (NIPALS):

1. Compute weight vectors $w=X^{T}Yc/\Vert X^{T}Yc\Vert$ and $c=Y^{T}Xw/\Vert Y^{T}Xw\Vert$ iteratively.
2. Compute scores: $t=Xw$ (X-scores), $u=Yc$ (Y-scores).
3. Compute loadings: $p=X^{T}t/(t^{T}t)$, $q=Y^{T}t/(t^{T}t)$.
4. Deflate: $X\leftarrow X-t{p}^T$, $Y\leftarrow Y-t{q}^T$.
5. Repeat for each component.

The regression coefficients are then: $\hat{B}=W(P^{T}W{)}^{-1}{Q}^T$.

Canonical Correlation Analysis (CCA): CCA finds pairs of linear combinations with maximum correlation:

$(w_k,c_k)=\arg \underset{w,c}{\max }\text{corr}(Xw,Yc)=\arg \underset{w,c}{\max }\frac{w^T{\mathrm{\Sigma }}_{XY}c}{\sqrt{w^T{\mathrm{\Sigma }}_{XX}w\cdot c^T{\mathrm{\Sigma }}_{YY}c}}$

The solution involves the generalized eigenvalue problem:

${\mathrm{\Sigma }}_{XX}^{-1}{\mathrm{\Sigma }}_{XY}{\mathrm{\Sigma }}_{YY}^{-1}{\mathrm{\Sigma }}_{YX}w={\rho }^{2}w$

where $\rho$ is the canonical correlation. CCA differs from PLS in that it maximizes correlation rather than covariance, making it invariant to scaling of the variables.

Comparison:

Method	Objective	Best for
PLS	Max covariance	Prediction, high-dimensional $X$
CCA	Max correlation	Finding shared structure, equal dimensionality

Related Pages

• Implementation:Scikit_learn_Scikit_learn_PLSRegression