Implementation:Scikit learn Scikit learn OptimizeUtils

Knowledge Sources	Scikit_learn Scikit-learn Docs
Domains	Machine Learning, Optimization
Last Updated	2026-02-08 15:00 GMT

Overview

Concrete utility module for Newton conjugate gradient optimization provided by scikit-learn.

Description

The optimize module provides scikit-learn's own implementation of the Newton conjugate gradient solver. Unlike scipy's version, it uses only one function call to retrieve the function value, gradient, and Hessian matrix-vector product, which provides significant speedups for expensive loss functions (e.g., logistic regression with large matrices). It also includes Wolfe line search and optimization result checking.

Usage

Use these functions when implementing custom loss functions that need efficient second-order optimization, or when checking the result of scipy optimization calls within scikit-learn estimators.

Code Reference

Source Location

• Repository: scikit-learn
• File: sklearn/utils/optimize.py

Signature

class _LineSearchError(RuntimeError):
    ...

def _line_search_wolfe12(
    f, fprime, xk, pk, gfk, old_fval, old_old_fval, verbose=0, **kwargs
):
    ...

def _cg(fhess_p, fgrad, maxiter, tol, verbose=0):
    ...

def _newton_cg(
    grad_hess, func, grad, x0, args=(), tol=1e-4, maxiter=100,
    maxinner=200, line_search=True, warn=True, verbose=0
):
    ...

def _check_optimize_result(solver, result, max_iter=None, extra_warning_msg=None):
    ...

Import

from sklearn.utils.optimize import _newton_cg, _check_optimize_result

I/O Contract

Inputs

Name	Type	Required	Description
grad_hess	callable	Yes	Function returning gradient and Hessian-vector product callable
func	callable	Yes	Objective function to minimize
grad	callable	Yes	Gradient function
x0	ndarray	Yes	Initial parameter vector
tol	float	No	Convergence tolerance
maxiter	int	No	Maximum number of Newton iterations

Outputs

Name	Type	Description
xk	ndarray	Optimized parameter vector
n_iter	int	Number of iterations performed

Usage Examples

Basic Usage

from sklearn.utils.optimize import _check_optimize_result
import scipy.optimize

# Check a scipy optimization result
result = scipy.optimize.minimize(lambda x: x**2, x0=1.0, method="L-BFGS-B")
_check_optimize_result("lbfgs", result)

Related Pages

• Principle:Scikit_learn_Scikit_learn_Generalized_Linear_Models