Workflow:Run llama Llama index Embedding Finetuning
| Knowledge Sources | |
|---|---|
| Domains | Embeddings, Fine_Tuning, Information_Retrieval |
| Last Updated | 2026-02-11 19:00 GMT |
Overview
End-to-end process for fine-tuning embedding models on domain-specific query-document pairs to improve retrieval quality in RAG pipelines.
Description
This workflow fine-tunes embedding models (using Sentence Transformers or a lightweight adapter layer) on custom query-document relevance pairs. The goal is to produce embeddings that place domain-relevant queries closer to their corresponding documents in vector space, improving retrieval accuracy for downstream RAG applications. The workflow supports two approaches: full Sentence Transformer fine-tuning (retraining the entire model) and adapter fine-tuning (training a small linear layer on top of a frozen base embedding model).
Usage
Execute this workflow when the default embedding model underperforms on domain-specific retrieval tasks. This is indicated by low recall or precision in retrieval evaluation, or when the domain vocabulary differs significantly from the embedding model's training distribution. Requires a dataset of queries mapped to relevant document passages.
Execution Steps
Step 1: Generate Training Dataset
Create an EmbeddingQAFinetuneDataset containing queries, a document corpus, and relevance mappings. This can be done manually by curating query-document pairs, or automatically using generate_qa_embedding_pairs() which uses an LLM to generate questions from document passages.
Key considerations:
- The dataset contains three components: queries (id-to-text), corpus (id-to-text), and relevant_docs (query-id-to-corpus-ids)
- LLM-generated questions provide a scalable way to create training data
- The dataset can be saved and loaded as JSON for reuse
- Consider creating a separate validation dataset for monitoring training progress
Step 2: Configure Fine-tuning Engine
Create a SentenceTransformersFinetuneEngine (for full model fine-tuning) or an EmbeddingAdapterFinetuneEngine (for adapter-only fine-tuning). Configure the base model, output path, training hyperparameters, and loss function.
Key considerations:
- SentenceTransformersFinetuneEngine retrains the full model; more expressive but slower
- EmbeddingAdapterFinetuneEngine trains only a small adapter; faster and uses less memory
- The default loss function is MultipleNegativesRankingLoss
- evaluation_steps controls how frequently validation metrics are computed
Step 3: Execute Fine-tuning
Call finetune() to start training. For Sentence Transformers, this runs the standard model.fit() training loop with the configured DataLoader, loss function, and evaluator. For the adapter, this trains a linear projection layer using gradient descent. The model is saved to the output path upon completion.
Key considerations:
- Training uses PyTorch DataLoader with configurable batch size
- The InformationRetrievalEvaluator computes MRR, MAP, and NDCG during training
- Training runs locally on available GPU hardware
- The output path contains the complete model files for loading
Step 4: Load Fine-tuned Model
Call get_finetuned_model() to create a LlamaIndex BaseEmbedding instance pointing to the fine-tuned model weights. For Sentence Transformers, this loads the full model; for adapters, this wraps the adapter layer around the original base model.
Key considerations:
- The returned model is a standard LlamaIndex embedding model
- It can be used anywhere a BaseEmbedding is accepted
- For adapters, the base model must still be available at the original path
Step 5: Integrate and Evaluate
Use the fine-tuned embedding model in a VectorStoreIndex or IngestionPipeline by setting it as the embed_model. Compare retrieval performance against the base model using evaluation metrics to validate improvement.
Key considerations:
- Set Settings.embed_model = finetuned_model for global usage
- Or pass embed_model= directly to VectorStoreIndex or pipeline
- Use the evaluation framework to measure retrieval quality before and after