Principle:Scikit learn Scikit learn Generalized Linear Models

Knowledge Sources	Scikit_learn Scikit-learn Docs
Domains	Supervised Learning, Statistical Modeling
Last Updated	2026-02-08 15:00 GMT

Overview

Generalized linear models extend ordinary linear regression by allowing the response variable to follow distributions from the exponential family and relating the mean to the linear predictor through a link function.

Description

Generalized Linear Models (GLMs) provide a unified framework for regression when the response variable does not follow a Gaussian distribution. They accommodate count data (Poisson), positive continuous data (Gamma), binary data (Bernoulli/Binomial), and other exponential family distributions. GLMs solve the problem of applying linear modeling principles to response variables that violate the normality assumption of ordinary least squares. They occupy a central role in statistical modeling, bridging classical linear regression with more flexible non-linear approaches.

Usage

Use GLMs when the response variable has a non-Gaussian distribution but a known relationship to the exponential family. Use PoissonRegressor for count data (e.g., number of events, insurance claims). Use GammaRegressor for positive continuous data that is right-skewed (e.g., insurance claim amounts, durations). Use TweedieRegressor when the response has a Tweedie distribution, which encompasses Poisson and Gamma as special cases and is particularly useful for data with exact zeros and a continuous positive component. GLMs are especially important in actuarial science, healthcare, and ecology.

Theoretical Basis

A GLM consists of three components:

1. Random component: The response variable $y$ follows a distribution from the exponential family:

   $p(y|\theta ,\varphi )=\exp (\frac{y\theta -b(\theta )}{a(\varphi )}+c(y,\varphi ))$

   where $\theta$ is the natural parameter, $\varphi$ is the dispersion parameter, and $b(\theta )$ is the cumulant function.

1. Systematic component: A linear predictor $\eta =X\beta$.

1. Link function: A monotonic function $g$ relating the conditional mean $\mu =E[y|X]$ to the linear predictor: $g(\mu )=\eta$.

Common GLM families and their canonical link functions:

Distribution	Link Function	$g(\mu )$
Gaussian	Identity	$\mu$
Poisson	Log	$\log (\mu )$
Gamma	Reciprocal	$1/\mu$
Bernoulli	Logit	$\log (\mu /(1-\mu ))$

Parameter estimation is performed by maximizing the log-likelihood, typically via Iteratively Reweighted Least Squares (IRLS) or Newton's method. With an ${\ell }_2$ penalty, the objective becomes:

$\hat{\beta }=\arg \underset{\beta }{\min }-\frac{1}{n}\sum _{i=1}^n\log p(y_i|x_i,\beta )+\frac{\alpha }{2}\Vert \beta {\Vert }_2^2$

The Tweedie distribution is a special case of the exponential family parameterized by a power parameter $p$:

• $p=0$: Gaussian
• $p=1$: Poisson
• $1<p<2$: compound Poisson-Gamma (zero-inflated continuous)
• $p=2$: Gamma
• $p=3$: Inverse Gaussian

Related Pages

• Implementation:Scikit_learn_Scikit_learn_GeneralizedLinearRegressor
• Implementation:Scikit_learn_Scikit_learn_NewtonSolver
• Implementation:Scikit_learn_Scikit_learn_LinearModelLoss