Environment:Ggml org Ggml Python Tooling Environment
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Model_Conversion |
| Last Updated | 2026-02-10 07:40 GMT |
Overview
Python 3 environment with PyTorch, TensorFlow, Transformers, and GGUF libraries for model conversion, weight extraction, and external model training scripts.
Description
This environment provides the Python tooling required to convert models from various deep learning frameworks (PyTorch, TensorFlow/Keras, HuggingFace) into GGML's binary format and GGUF format. It includes conversion scripts for GPT-2, GPT-J, SAM, YOLOv3-tiny, MNIST, and Magika models. The environment is also needed for running external training scripts (MNIST FC via PyTorch, MNIST CNN via TensorFlow/Keras) that produce weights for GGML evaluation.
Usage
Use this environment when you need to convert models from PyTorch/TensorFlow/HuggingFace formats to GGML/GGUF binary format, or when training models externally using the provided Python scripts. This environment is not required for running inference with pre-converted models.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Linux, macOS, or Windows | Cross-platform Python |
| Python | Python 3.x | Python 3.10+ recommended |
| Disk | 2GB+ | For Python packages and model files |
Dependencies
Python Packages
- `torch` ~= 2.5.1 (CPU wheels from pytorch.org)
- `torchvision` >= 0.15.2
- `tensorflow` == 2.18.0
- `keras` == 3.5.0
- `transformers` >= 4.35.2, < 5.0.0
- `accelerate` == 0.19.0
- `numpy` >= 2.0.2
- `sentencepiece` ~= 0.1.98
- `gguf` >= 0.1.0
Credentials
No credentials are required for model conversion. However, some HuggingFace models may require:
- `HF_TOKEN`: HuggingFace API token for gated model access (only needed for restricted models).
Quick Install
# Install all required packages
pip install -r requirements.txt
# Or install individually
pip install torch torchvision tensorflow keras transformers accelerate numpy sentencepiece gguf
Code Evidence
Full dependency specification from `requirements.txt:1-11`:
accelerate==0.19.0
numpy>=2.0.2
sentencepiece~=0.1.98
torchvision>=0.15.2
transformers>=4.35.2,<5.0.0
gguf>=0.1.0
keras==3.5.0
tensorflow==2.18.0
--extra-index-url https://download.pytorch.org/whl/cpu
torch~=2.5.1
PyTorch usage in GPT-2 H5 conversion from `examples/gpt-2/convert-h5-to-ggml.py:1-10`:
import sys
import struct
import json
import numpy as np
from transformers import GPT2Model
TensorFlow usage in MNIST CNN training from `examples/mnist/mnist-train-cnn.py:1-5`:
import gguf
import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `ModuleNotFoundError: No module named 'transformers'` | Transformers not installed | `pip install transformers>=4.35.2` |
| `ModuleNotFoundError: No module named 'gguf'` | GGUF package not installed | `pip install gguf>=0.1.0` |
| `ImportError: cannot import name 'GPT2Model'` | Incompatible transformers version | `pip install transformers>=4.35.2,<5.0.0` |
| `torch version mismatch` | Wrong PyTorch version | `pip install torch~=2.5.1 --extra-index-url https://download.pytorch.org/whl/cpu` |
Compatibility Notes
- CPU-only PyTorch: The requirements install CPU-only torch wheels by default (via `--extra-index-url`). GPU PyTorch is not required for conversion scripts.
- TensorFlow version: Pinned to 2.18.0 for Keras 3.5.0 compatibility. Mixing versions may cause import errors.
- Conversion scripts: Each example directory has its own conversion script(s) that may use different subsets of these dependencies.