Implementation:AnswerDotAI RAGatouille Export To Huggingface Hub
| Knowledge Sources | |
|---|---|
| Domains | Model_Export, Model_Distribution, NLP |
| Last Updated | 2026-02-12 12:00 GMT |
Overview
Concrete tool for uploading trained ColBERT model checkpoints to the Hugging Face Hub provided by the RAGatouille library.
Description
The export_to_huggingface_hub function packages a local ColBERT model checkpoint and uploads it to a Hugging Face Hub repository. It validates the checkpoint contains a valid ColBERTConfig, optionally re-saves the model to a temporary directory for clean export, and can also produce a Vespa-compatible ONNX file alongside the upload. The function handles authentication errors and repository creation automatically via the HfApi client.
Usage
Use this function after training or fine-tuning a ColBERT model when you want to distribute the model publicly or privately via the Hugging Face Hub. It is typically the final step in a training workflow, enabling other users to load the model with RAGPretrainedModel.from_pretrained().
Code Reference
Source Location
- Repository: AnswerDotAI_RAGatouille
- File: ragatouille/models/utils.py
- Lines: 25-97
Signature
def export_to_huggingface_hub(
colbert_path: Union[str, Path],
huggingface_repo_name: str,
export_vespa_onnx: bool = False,
use_tmp_dir: bool = False,
) -> None:
"""
Upload a ColBERT model checkpoint to the Hugging Face Hub.
Args:
colbert_path: Local path to the ColBERT model checkpoint directory.
huggingface_repo_name: Target repository name on HF Hub (e.g. 'username/model-name').
export_vespa_onnx: If True, also export and upload a Vespa ONNX model.
use_tmp_dir: If True, re-save model to a temp directory before uploading.
"""
Import
from ragatouille.models.utils import export_to_huggingface_hub
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| colbert_path | Union[str, Path] | Yes | Local filesystem path to the ColBERT checkpoint directory |
| huggingface_repo_name | str | Yes | Target Hugging Face Hub repository ID (format: 'username/repo-name') |
| export_vespa_onnx | bool | No | Whether to also export a Vespa ONNX model alongside the checkpoint (default: False) |
| use_tmp_dir | bool | No | Whether to re-save the model to a temporary directory before upload (default: False) |
Outputs
| Name | Type | Description |
|---|---|---|
| return | None | No return value; prints status messages to stdout |
| side effect | HF Hub upload | Model files are uploaded to the specified Hugging Face Hub repository |
| side effect | Repo creation | Creates the repository on HF Hub if it does not exist |
Usage Examples
Basic Upload
from ragatouille.models.utils import export_to_huggingface_hub
# Upload a trained ColBERT model to Hugging Face Hub
export_to_huggingface_hub(
colbert_path="experiments/colbert/none/2024-01/checkpoints/colbert",
huggingface_repo_name="myuser/my-colbert-model",
)
Upload with Vespa ONNX Export
from ragatouille.models.utils import export_to_huggingface_hub
# Upload model AND include a Vespa-compatible ONNX export
export_to_huggingface_hub(
colbert_path="experiments/colbert/none/2024-01/checkpoints/colbert",
huggingface_repo_name="myuser/my-colbert-model",
export_vespa_onnx=True,
use_tmp_dir=True, # Re-save to temp dir for clean export
)