Implementation:Rapidsai Cuml Batched LBFGS

Knowledge Sources	Rapidsai_Cuml
Domains	Machine_Learning, Optimization, Time_Series
Last Updated	2026-02-08 12:00 GMT

Overview

Implements a batch-aware L-BFGS-B optimizer that simultaneously minimizes a loss function across multiple independent time series, using SciPy's low-level Fortran/C L-BFGS-B routine.

Description

The batched_fmin_lbfgs_b function provides a batched variant of the L-BFGS-B (Limited-memory Broyden-Fletcher-Goldfarb-Shanno with Box constraints) optimization algorithm. It is designed for the ARIMA time series module in cuML, where the same model structure must be fit to many independent time series simultaneously.

The implementation wraps SciPy's internal _lbfgsb.setulb Fortran routine, maintaining separate working arrays (x, f, g, wa, iwa, task, etc.) for each batch member. In each iteration:

1. The SciPy routine updates each batch member's state (request function evaluation, report convergence, etc.).
2. The batched function and gradient are evaluated once on the concatenated parameter vector.
3. Results are dispatched back to each batch member.

The module handles compatibility with SciPy >= 1.15 (which switched to a C-based L-BFGS-B implementation with integer task codes) and earlier versions (which used string-based task codes).

When no gradient function (fprime) is provided, the internal _fd_fprime helper computes a central finite-difference approximation.

Usage

This function is used internally by cuML's ARIMA and AutoARIMA models to optimize log-likelihood parameters for batches of time series. It is not typically called directly by end users but is the core optimizer behind batched ARIMA fitting.

Code Reference

Source Location

• Repository: Rapidsai_Cuml
• File: python/cuml/cuml/tsa/batched_lbfgs.py

Signature

def batched_fmin_lbfgs_b(
    func,
    x0,
    num_batches,
    fprime=None,
    args=(),
    bounds=None,
    m=10,
    factr=1e7,
    pgtol=1e-5,
    epsilon=1e-8,
    iprint=-1,
    maxiter=15000,
    maxls=20,
)

Import

from cuml.tsa.batched_lbfgs import batched_fmin_lbfgs_b

I/O Contract

Inputs

Name	Type	Required	Description
func	callable: array -> array[M]	Yes	Loss function to minimize. Takes a concatenated parameter vector and returns an array of M loss values (one per batch member).
x0	array of shape (M * n_params,)	Yes	Starting parameters for all batches, concatenated.
num_batches	int	Yes	Number of independent optimization problems (M).
fprime	callable: array -> array[M * n_params]	No	Gradient function. Returns derivatives concatenated across batches. If None, uses central finite differences.
args	tuple	No	Additional arguments passed to func and fprime. Default ().
bounds	list of (float, float) tuples	No	Box constraints on each parameter. Length must equal n_params. None values indicate no bound.
m	int	No	Number of previous gradient vectors stored for inverse Hessian approximation. Default 10.
factr	float	No	f(xk+1) - f(xk)| < factr * eps_mach. Default 1e7.
pgtol	float	No	grad| < pgtol. Default 1e-5.
epsilon	float	No	Finite differencing step size when approximating the gradient. Default 1e-8.
iprint	int	No	Verbosity level. -1 = silent. Only used for SciPy < 1.15. Default -1.
maxiter	int	No	Maximum number of L-BFGS iterations. Default 15000.
maxls	int	No	Maximum number of line-search iterations per step. Default 20.

Outputs

Name	Type	Description
xk	ndarray of shape (M * n_params,)	Optimized parameters for all batches, concatenated.
n_iterations	ndarray of shape (M,)	Number of iterations each batch member took to converge.
warn_flag	ndarray of shape (M,)	Convergence status per batch: 0 = converged, 2 = abnormal termination.

Usage Examples

import numpy as np
from cuml.tsa.batched_lbfgs import batched_fmin_lbfgs_b

# Minimize a simple quadratic for 2 independent batches
# f(x) = (x - target)^2 for each batch
targets = np.array([3.0, 7.0])

def func(x):
    # x has 2 parameters (one per batch)
    return (x - targets) ** 2

def fprime(x):
    return 2.0 * (x - targets)

x0 = np.array([0.0, 0.0])
xk, n_iter, warn = batched_fmin_lbfgs_b(
    func, x0, num_batches=2, fprime=fprime, maxiter=100
)
print("Optimized:", xk)    # close to [3.0, 7.0]
print("Iterations:", n_iter)
print("Warnings:", warn)

Related Pages

• Environment:Rapidsai_Cuml_CUDA_GPU
• Environment:Rapidsai_Cuml_Python_RAPIDS_Stack