Implementation:Scikit learn Scikit learn NewtonSolver

Knowledge Sources	Scikit_learn Scikit-learn Docs
Domains	Machine Learning, Numerical Optimization
Last Updated	2026-02-08 15:00 GMT

Overview

Concrete tool for second-order Newton optimization of Generalized Linear Models provided by scikit-learn.

Description

NewtonSolver is an abstract base class implementing Newton/second-order optimization routines for GLMs. Each Newton iteration solves H @ coef_newton = -g where H is the Hessian and g is the gradient. For LinearModelLoss, the gradient is X.T @ loss.gradient + l2_reg_strength * coef and the Hessian is X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity. A backtracking line search is used to update coefficients. The concrete subclass NewtonCholeskySolver uses Cholesky-based linear solvers for the inner Newton step.

Usage

NewtonSolver is an internal solver used by scikit-learn's GLM estimators (PoissonRegressor, GammaRegressor, TweedieRegressor) when the solver parameter is set to 'newton-cholesky'. It is not typically instantiated directly by users but provides the optimization backbone for GLM fitting.

Code Reference

Source Location

• Repository: scikit-learn
• File: sklearn/linear_model/_glm/_newton_solver.py

Signature

class NewtonSolver(ABC):
    def __init__(
        self,
        *,
        coef,
        linear_loss=LinearModelLoss(base_loss=HalfSquaredError(), fit_intercept=True),
        l2_reg_strength=0.0,
        tol=1e-4,
        max_iter=100,
        n_threads=1,
        verbose=0,
    ):

class NewtonCholeskySolver(NewtonSolver):
    """Cholesky based Newton solver."""

Import

from sklearn.linear_model._glm._newton_solver import NewtonSolver, NewtonCholeskySolver

I/O Contract

Inputs

Name	Type	Required	Description
coef	ndarray of shape (n_dof,)	Yes	Initial coefficients of the linear model
linear_loss	LinearModelLoss	No	The loss to be minimized (default: squared error)
l2_reg_strength	float	No	L2 regularization strength (default=0.0)
tol	float	No	Convergence tolerance for gradient and Newton decrement (default=1e-4)
max_iter	int	No	Maximum number of Newton steps allowed (default=100)
n_threads	int	No	Number of OpenMP threads for Hessian/gradient computation (default=1)
verbose	int	No	Verbosity level (default=0)

Outputs

Name	Type	Description
coef	ndarray	Optimized coefficient vector after solving
n_iter_	int	Number of Newton iterations performed
converged	bool	Whether the optimization converged within tolerance

Usage Examples

Basic Usage

# NewtonSolver is typically used internally by GLM estimators.
# Direct usage via GLM estimators:
from sklearn.linear_model import PoissonRegressor
import numpy as np

X = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
y = np.array([1, 2, 3, 4])
model = PoissonRegressor(solver="newton-cholesky")
model.fit(X, y)
print("Coefficients:", model.coef_)

Related Pages

• Principle:Scikit_learn_Scikit_learn_Generalized_Linear_Models