Implementation:Ggml org Llama cpp Fit Params

Knowledge Sources	Ggml_org_Llama_cpp
Domains	Parameter_Estimation
Last Updated	2026-02-15 00:00 GMT

Overview

CLI utility that determines optimal model and context parameters to fit within available device memory and prints the corresponding CLI arguments to stdout.

Description

This tool parses common parameters, initializes the llama backend, then calls `llama_params_fit()` with the model path, model/context params, tensor split, and buffer type overrides. On success, it prints the fitted `-c` (context size), `-ngl` (GPU layers), `-ts` (tensor split), and `-ot` (tensor buffer type overrides) arguments to stdout. These arguments can be piped directly into other llama.cpp tools like llama-server for automatic memory-optimal configuration.

Usage

Use this tool to automatically calculate the optimal GPU layer offloading, context size, and tensor split configuration for a given model and hardware setup, eliminating the need for manual GPU memory estimation.

Code Reference

Source Location

• Repository: Ggml_org_Llama_cpp
• File: tools/fit-params/fit-params.cpp
• Lines: 1-66

Signature

int main(int argc, char ** argv);

Import

#include "llama.h"
#include "arg.h"
#include "common.h"
#include "log.h"

I/O Contract

Inputs

Name	Type	Required	Description
-m	string (CLI arg)	Yes	Path to the GGUF model file
-c	int (CLI arg)	No	Desired context size (used as starting point)
-ngl	int (CLI arg)	No	Desired number of GPU layers (used as starting point)
-ts	float list (CLI arg)	No	Initial tensor split configuration
-ot	string (CLI arg)	No	Initial tensor buffer type overrides

Outputs

Name	Type	Description
stdout	text	Fitted CLI arguments: -c <ctx_size> -ngl <gpu_layers> [-ts <splits>] [-ot <overrides>]

Usage Examples

# Determine optimal parameters for a model
./llama-fit-params -m model.gguf

# Pipe results directly into llama-server
./llama-server -m model.gguf $(./llama-fit-params -m model.gguf)

Related Pages

• Principle:Ggml_org_Llama_cpp_Parameter_Estimation