Implementation:Ggml org Llama cpp Multimodal Download
| Aspect | Detail |
|---|---|
| Implementation Name | Multimodal Download |
| Doc Type | External Tool Doc |
| Domain | Multimodal Inference |
| Purpose | Downloading paired text model GGUF and mmproj GGUF files |
| Related Workflow | Multimodal_Inference |
Overview
Description
This implementation documents the process of downloading the two GGUF files required for multimodal inference in llama.cpp: the text model GGUF and the multimodal projector (mmproj) GGUF. These files are typically hosted on Hugging Face model repositories and can be downloaded using command-line tools such as wget, curl, or the huggingface-cli.
Usage
Multimodal model files must be downloaded before any multimodal inference session. The user must ensure that both files come from the same model family and are compatible with each other. Typical sources include:
- Hugging Face Hub: Most multimodal GGUF models are published under organization pages (e.g.,
ggml-org, model author pages) - Direct URLs: Some model providers offer direct download links
Code Reference
| Aspect | Detail |
|---|---|
| Source Location | External tool (Hugging Face Hub, wget, curl, huggingface-cli) |
| Signature | N/A (external download process) |
| Import | N/A |
The download process is external to llama.cpp itself. The project does not include built-in download scripts for multimodal models, but the general pattern involves retrieving two files from the same model repository.
I/O Contract
| Direction | Name | Type | Description |
|---|---|---|---|
| Input | Model repository URL | String | URL to a Hugging Face repository or direct download link |
| Input | Model variant | String | Quantization level for the text model (e.g., Q4_K_M, Q8_0) |
| Output | Text model GGUF file | File | The quantized language model (e.g., model-Q4_K_M.gguf)
|
| Output | mmproj GGUF file | File | The multimodal projector (e.g., mmproj-f16.gguf)
|
Usage Examples
Example 1: Download using wget
# Download a text model GGUF
wget https://huggingface.co/<org>/<model>/resolve/main/<model>-Q4_K_M.gguf
# Download the matching multimodal projector
wget https://huggingface.co/<org>/<model>/resolve/main/mmproj-f16.gguf
Example 2: Download using huggingface-cli
# Download specific files from a repository
huggingface-cli download <org>/<model> <model>-Q4_K_M.gguf mmproj-f16.gguf --local-dir ./models/
Example 3: Verify downloaded files
# Check file sizes to ensure complete downloads
ls -lh models/*.gguf
Key considerations:
- The mmproj file is typically distributed at F16 precision regardless of the text model quantization level
- Both files must come from the same model family (e.g., both from LLaVA-1.6, both from Gemma-3)
- File naming conventions vary by model provider, but mmproj files commonly contain "mmproj" in their filename