Implementation:Ggml org Llama cpp Finetune
| Knowledge Sources | |
|---|---|
| Domains | Training, Fine_Tuning |
| Last Updated | 2026-02-15 00:00 GMT |
Overview
Implements language model finetuning using GGML's optimization framework directly within llama.cpp.
Description
This example program loads a model with memory mapping disabled (required for writable weights) and forces F32 KV cache types due to the lack of F16 OUT_PROD support. It tokenizes a training file, creates an optimization dataset with half-context-length sequences, then runs the GGML optimizer to minimize cross-entropy loss over multiple epochs. After training, the finetuned model is saved to disk.
Usage
Use this program as an experimental proof-of-concept for finetuning models directly within the llama.cpp/GGML ecosystem, enabling training without external frameworks like PyTorch. It is invoked as a standalone CLI executable with common llama.cpp parameters.
Code Reference
Source Location
- Repository: Ggml_org_Llama_cpp
- File: examples/training/finetune.cpp
- Lines: 1-97
Signature
int main(int argc, char ** argv);
Import
#include "arg.h"
#include "common.h"
#include "log.h"
#include "llama.h"
#include <cmath>
#include <cstdio>
#include <cstring>
#include <ctime>
#include <vector>
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| argc | int | Yes | Command-line argument count |
| argv | char ** | Yes | Command-line arguments including model path, training file (--prompt), learning rate, epochs, etc. |
| params.prompt | std::string | Yes | Training text to tokenize and use as the finetuning dataset |
| params.lr | lr_opt | No | Learning rate schedule parameters (lr0, wd, lr_min, decay_epochs, epochs) |
| params.out_file | std::string | No | Output file path for the finetuned model |
Outputs
| Name | Type | Description |
|---|---|---|
| return value | int | 0 on success, 1 on failure |
| model file | file | Finetuned model saved to params.out_file via llama_model_save_to_file |
Usage Examples
# Finetune a model on a text file
./llama-finetune -m model.gguf --prompt "$(cat training_data.txt)" --out model-finetuned.gguf --epochs 5