Principle:AnswerDotAI RAGatouille Model Training

Knowledge Sources	ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction RAGatouille
Domains	NLP, Information_Retrieval, Training, Fine_Tuning
Last Updated	2026-02-12 12:00 GMT

Overview

A training execution mechanism that fine-tunes or trains a ColBERT model on prepared triplet data using contrastive learning with optional in-batch negatives.

Description

Model Training is the final step in the ColBERT training pipeline. It takes prepared training data (triplets of query, positive, negative) and a configured set of hyperparameters, then runs the ColBERT training loop. The training uses the colbert-ai Trainer class which handles the optimization loop, gradient accumulation, learning rate scheduling, checkpoint saving, and in-batch negative computation.

The training process:

• Constructs ColBERTConfig from user-provided hyperparameters
• Initializes the colbert-ai Trainer with training data files (triples, queries, corpus)
• Runs the training loop with automatic checkpointing
• Returns the path to the best checkpoint

Usage

Use this principle as the final step after preparing training data. The trained model can then be used with RAGPretrainedModel.from_pretrained() for indexing and search.

Theoretical Basis

ColBERT training minimizes a contrastive loss over triplets:

${ℒ}=-\log \frac{e^{S(q,d^{+})}}{e^{S(q,d^{+})}+\sum _{d^{-}\in N}{e}^{S(q,d^{-})}}$

With in-batch negatives enabled (use_ib_negatives=True), the negative set N includes both the explicit negative from the triplet and all other documents in the batch, providing stronger training signal.

Training Loop:

1. Sample a batch of (query, positive, negative) triplets
2. Encode queries and documents with the ColBERT encoder
3. Compute MaxSim scores for all query-document pairs
4. Compute cross-entropy loss
5. Update model weights via backpropagation
6. Save checkpoints at configured intervals

nway=2: RAGatouille fixes the number of ways to 2 (one positive, one negative per triplet), relying on in-batch negatives for additional contrast.

Related Pages

Implemented By

• Implementation:AnswerDotAI_RAGatouille_RAGTrainer_Train