Implementation:Scikit learn Scikit learn LinearModelLoss
| Knowledge Sources | |
|---|---|
| Domains | Machine Learning, Loss Functions |
| Last Updated | 2026-02-08 15:00 GMT |
Overview
Concrete tool for computing loss, gradient, and Hessian for linear models with raw_prediction = X @ coef + intercept provided by scikit-learn.
Description
LinearModelLoss is a general-purpose class for loss functions where the raw prediction is a linear function of the features. It computes the average per-sample loss with optional L2 regularization: loss = 1/s_sum * sum(s_i * loss(y_i, X_i @ coef + intercept)) + 1/2 * l2_reg_strength * ||coef||^2. The class provides methods for computing the loss value, gradient, and Hessian matrix, supporting both single-class and multi-class settings. It also includes the sandwich_dot utility for computing X.T @ diag(W) @ X efficiently.
Usage
LinearModelLoss is an internal utility class used by scikit-learn's solvers (Newton solvers, L-BFGS) when fitting linear models. It provides a unified interface for loss computation across different loss functions (squared error, logistic, Poisson, etc.) and is not typically used directly by end users.
Code Reference
Source Location
- Repository: scikit-learn
- File: sklearn/linear_model/_linear_loss.py
Signature
class LinearModelLoss:
"""General class for loss functions with raw_prediction = X @ coef + intercept."""
def __init__(self, base_loss, fit_intercept):
def sandwich_dot(X, W):
"""Compute the sandwich product X.T @ diag(W) @ X."""
Import
from sklearn.linear_model._linear_loss import LinearModelLoss
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| base_loss | BaseLoss instance | Yes | The per-sample loss function from sklearn._loss |
| fit_intercept | bool | Yes | Whether the model includes an intercept term |
Outputs
| Name | Type | Description |
|---|---|---|
| loss | float | Computed average loss value with optional L2 regularization |
| gradient | ndarray | Gradient of the loss with respect to coefficients (shape follows coef) |
| hessian | ndarray | Hessian matrix of the loss (shape (n_dof, n_dof) or (n_classes*n_dof, n_classes*n_dof)) |
Usage Examples
Basic Usage
# LinearModelLoss is an internal class used by solvers.
# Typical indirect usage via a GLM estimator:
from sklearn.linear_model import PoissonRegressor
import numpy as np
X = np.array([[1, 2], [3, 4], [5, 6]])
y = np.array([1, 2, 3])
model = PoissonRegressor()
model.fit(X, y)
print("Coefficients:", model.coef_)