Implementation:Haifengl Smile LevenbergMarquardt

Knowledge Sources	Haifengl_Smile
Domains	Mathematics, Optimization, Curve Fitting
Last Updated	2026-02-08 22:00 GMT

Overview

LevenbergMarquardt is a Java record that implements the Levenberg-Marquardt algorithm for solving non-linear least squares curve-fitting problems.

Description

The LevenbergMarquardt record in the smile.math package implements the damped least-squares method (LMA) for generic curve-fitting. The algorithm interpolates between the Gauss-Newton algorithm (GNA) and the method of gradient descent, making it more robust than pure GNA. It uses SVD (Singular Value Decomposition) internally to solve the normal equations at each iteration.

As a Java record, the result object exposes four components:

• parameters -- the fitted parameter values
• fittedValues -- the model predictions at each data point
• residuals -- the difference between observed and fitted values
• sse -- the sum of squared errors

The algorithm finds only a local minimum, not necessarily the global minimum.

Usage

Use LevenbergMarquardt when you need to fit a parametric nonlinear model to observed data, such as fitting exponential decay curves, Gaussian peaks, or other nonlinear regression models. You must provide a DifferentiableMultivariateFunction that computes both the function value and partial derivatives with respect to the parameters.

Code Reference

Source Location

• Repository: Haifengl_Smile
• File: base/src/main/java/smile/math/LevenbergMarquardt.java
• Lines: 1-379

Signature

public record LevenbergMarquardt(
    double[] parameters,
    double[] fittedValues,
    double[] residuals,
    double sse
) {
    // Fit with default tolerance (0.0001) and max iterations (20)
    public static LevenbergMarquardt fit(
        DifferentiableMultivariateFunction func,
        double[] x, double[] y, double[] p);

    // Fit with custom tolerance and max iterations
    public static LevenbergMarquardt fit(
        DifferentiableMultivariateFunction func,
        double[] x, double[] y, double[] p,
        double stol, int maxIter);

    // Multivariate independent variable versions
    public static LevenbergMarquardt fit(
        DifferentiableMultivariateFunction func,
        double[][] x, double[] y, double[] p);

    public static LevenbergMarquardt fit(
        DifferentiableMultivariateFunction func,
        double[][] x, double[] y, double[] p,
        double stol, int maxIter);
}

Import

import smile.math.LevenbergMarquardt;

I/O Contract

Inputs

Name	Type	Required	Description
func	DifferentiableMultivariateFunction	Yes	The curve function that computes both value and gradient. The first d elements of the input are hyperparameters to fit; the rest is the independent variable.
x	double[] or double[][]	Yes	The independent variable values (univariate or multivariate)
y	double[]	Yes	The observed dependent variable values
p	double[]	Yes	The initial parameter estimates
stol	double	No	The scalar tolerance on fractional improvement in sum of squares (default: 0.0001)
maxIter	int	No	The maximum number of iterations (default: 20)

Outputs

Name	Type	Description
parameters	double[]	The fitted parameter values
fittedValues	double[]	The model predictions at each observation point
residuals	double[]	The residuals (y - fittedValues) at each point
sse	double	The sum of squared errors

Usage Examples

Basic Curve Fitting

// Define a differentiable function: y = a * exp(b * x)
// where p[0] = a, p[1] = b, p[2] = x
DifferentiableMultivariateFunction func = new DifferentiableMultivariateFunction() {
    @Override
    public double f(double[] p) {
        return p[0] * Math.exp(p[1] * p[2]);
    }

    @Override
    public double g(double[] p, double[] gradient) {
        double val = p[0] * Math.exp(p[1] * p[2]);
        gradient[0] = Math.exp(p[1] * p[2]);        // df/da
        gradient[1] = p[0] * p[2] * Math.exp(p[1] * p[2]); // df/db
        return val;
    }
};

double[] x = {0.0, 1.0, 2.0, 3.0, 4.0, 5.0};
double[] y = {1.0, 2.7, 7.4, 20.1, 54.6, 148.4};
double[] initialParams = {1.0, 1.0};

LevenbergMarquardt result = LevenbergMarquardt.fit(func, x, y, initialParams);
System.out.println("Fitted a = " + result.parameters()[0]);
System.out.println("Fitted b = " + result.parameters()[1]);
System.out.println("SSE = " + result.sse());

Custom Tolerance and Iterations

LevenbergMarquardt result = LevenbergMarquardt.fit(
    func, x, y, initialParams, 1e-6, 100);

Related Pages

• Implementation:Haifengl_Smile_MathEx