Implementation:Ggml org Llama cpp Conversion Utils
| Knowledge Sources | |
|---|---|
| Domains | Model_Conversion, Utilities |
| Last Updated | 2026-02-15 00:00 GMT |
Overview
Shared utility library for the model conversion scripts, providing common functions for data I/O, debugging, and validation.
Description
Key functions include: `save_output_data` for saving logits/embeddings as binary and text files alongside token and prompt metadata; `compare_tokens` for verifying tokenization consistency between PyTorch and llama.cpp models; `summarize` for printing tensors in llama.cpp debug format; `setup_rope_debug` for monkey-patching RoPE functions to dump activations; and `exit_with_warning` for diagnosing transformers version mismatches. Also provides `get_model_name_from_env_path` for resolving model names from environment variables.
Usage
Use this as the central utility module that all model conversion validation scripts depend on for consistent data serialization, token comparison, and debugging capabilities.
Code Reference
Source Location
- Repository: Ggml_org_Llama_cpp
- File: examples/model-conversion/scripts/utils/common.py
- Lines: 1-299
Signature
def get_model_name_from_env_path(env_path_name)
def summarize(tensor: torch.Tensor, name: str, max_seq: int = 3, max_vals: int = 3)
def debug_hook(name)
def setup_rope_debug(model)
def save_output_data(data, tokens, prompt, model_name, prefix, data_dir="data")
def compare_tokens(pytorch_tokens, llamacpp_tokens)
def show_version_warning(model_path)
def get_model_transformers_version(model_path)
def exit_with_warning(model_path)
Import
import os
import sys
import torch
import transformers
import json
import textwrap
import numpy as np
from pathlib import Path
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| env_path_name | str | Yes | Name of the environment variable containing the model path |
| tensor | torch.Tensor | Yes (for summarize) | Tensor to print in debug format (2D, 3D, or 4D) |
| data | np.ndarray | Yes (for save_output_data) | Logits or embeddings array to save |
| tokens | list | Yes (for save_output_data) | Token IDs associated with the output data |
| prompt | str | Yes (for save_output_data) | Prompt text used to generate the data |
| model_name | str | Yes (for save_output_data) | Name identifier for the model |
| prefix | str | Yes (for save_output_data) | Filename prefix (e.g., "pytorch" or "llamacpp") |
| pytorch_tokens | list | Yes (for compare_tokens) | Token IDs from PyTorch model |
| llamacpp_tokens | list | Yes (for compare_tokens) | Token IDs from llama.cpp model |
Outputs
| Name | Type | Description |
|---|---|---|
| get_model_name_from_env_path return | str | Model name extracted from the environment variable path |
| save_output_data files | files | Binary (.bin), text (.txt), tokens (.json), and prompt (.txt) files in data_dir |
| compare_tokens return | bool | True if tokens match, False otherwise |
Usage Examples
from common import get_model_name_from_env_path, save_output_data, compare_tokens
# Get model name from environment
model_name = get_model_name_from_env_path('MODEL_PATH')
# Save output data for later comparison
save_output_data(logits, tokens, prompt, model_name, prefix="pytorch")
# Compare tokenization between two models
tokens_match = compare_tokens(pytorch_tokens, llamacpp_tokens)