Implementation:Ggml org Llama cpp Debug
| Knowledge Sources | |
|---|---|
| Domains | Debugging, Tensor |
| Last Updated | 2026-02-15 00:00 GMT |
Overview
Implements tensor debugging utilities for printing tensor data and detecting NaN values during computation graph evaluation.
Description
The `common_debug_print_tensor` function iterates over a tensor's multi-dimensional data (up to 4D), reading values via type-aware accessors (F16, F32, BF16, I8, I16, I32, I64) and printing them in a nested bracket format with truncation for large dimensions (showing first/last n elements with "..." in between). It computes and displays the sum of all values and optionally aborts on NaN detection controlled by the template parameter. The `common_debug_cb_eval` function serves as a `ggml_backend_sched_eval_callback`, copying tensor data from the backend and printing it, with optional regex-based tensor name filtering.
Usage
Use this module during development and debugging to inspect intermediate tensor values during model inference. It is useful for diagnosing numerical issues such as NaN propagation and for verifying model correctness by examining tensor contents at each computation step.
Code Reference
Source Location
- Repository: Ggml_org_Llama_cpp
- File: common/debug.cpp
- Lines: 1-167
Signature
template <bool abort>
void common_debug_print_tensor(uint8_t * data, ggml_type type, const int64_t * ne, const size_t * nb, int64_t n);
template <bool abort_on_nan>
bool common_debug_cb_eval(struct ggml_tensor * t, bool ask, void * user_data);
Import
#include "debug.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| data | uint8_t * | Yes | Pointer to the raw tensor data bytes |
| type | ggml_type | Yes | The tensor's quantization/data type (F16, F32, BF16, I8, I16, I32, I64) |
| ne | const int64_t * | Yes | Array of tensor dimensions (up to 4D) |
| nb | const size_t * | Yes | Array of tensor strides in bytes |
| n | int64_t | Yes | Number of rows/columns to fully print before truncating |
| t | struct ggml_tensor * | Yes | The tensor to evaluate in the callback |
| ask | bool | Yes | If true, callback returns whether it wants to inspect this tensor |
| user_data | void * | Yes | Pointer to base_callback_data with filters and data buffer |
Outputs
| Name | Type | Description |
|---|---|---|
| cb_eval return | bool | True if the tensor should be evaluated (when ask=true), or true to continue evaluation |
| stdout output | formatted text | Nested bracket representation of tensor data with sum |
Usage Examples
#include "debug.h"
// Set up callback data with regex filters
base_callback_data cb_data(params, {"attn_.*", "ffn_.*"});
// The callback is automatically registered on params
// During inference, tensors matching the filters will be printed
// Direct tensor printing (3 elements before truncation)
common_debug_print_tensor<true>(tensor_data, GGML_TYPE_F32, ne, nb, 3);