Principle:FlagOpen FlagEmbedding LLM Reranker Training

Knowledge Sources	FlagOpen_FlagEmbedding
Domains	Machine Learning, Large Language Models, Information Retrieval, Reranking
Last Updated	2026-02-09 00:00 GMT

Overview

Training LLM-based rerankers using instruction-following and layer-wise approaches that leverage language models' understanding capabilities to refine retrieval results through pointwise or listwise scoring.

Description

This principle adapts large language models for reranking tasks where candidate documents retrieved by a first-stage retriever are scored and reordered based on relevance to a query. Unlike embedding-based retrievers that compute similarity in fixed vector spaces, LLM rerankers process the full query-document text through the language model and generate relevance scores via classification heads or language modeling probabilities. The approach supports two paradigms: instruction-following reranking where the model receives explicit prompts like "How relevant is this document to the query?", and layer-wise reranking that extracts scores from intermediate transformer layers for efficiency. Training uses pairwise or listwise ranking losses on labeled preference data. The method benefits from LLMs' deep language understanding, handling complex reasoning about relevance, but requires more computation than embedding similarity.

Usage

Use this principle when:

• Reranking retrieval results for improved precision
• Building second-stage rankers for search systems
• Leveraging LLM reasoning for relevance judgment
• Implementing layer-wise early exit for efficient reranking

Theoretical Basis

The LLM reranker training framework consists of:

1. Architecture Options:

• • Instruction-based:
    • Input: "Query: {q} Document: {d} Relevant: [Yes/No]"
    • Score: s = P(Yes | query, document)
    • Extract from classification head or token probability

• • Layer-wise:
    • Extract representations from multiple layers
    • Score from each layer: s_l = h_l · w_l
    • Enable early exit for efficiency

1. Training Objectives:

• • Pointwise: Binary classification
    • L = -log P(relevant | q, d+) - log P(not_relevant | q, d-)

• • Pairwise: Preference learning
    • L = -log σ(s(q, d+) - s(q, d-))

• • Listwise: Optimize ranking metrics directly
    • L = -Σ_i log(exp(s_i) / Σ_j exp(s_j))

1. Layer-wise Training:

• • Self-distillation: Train early layers to mimic final layer
  • L_distill = Σ_l KL(s_l || s_L)
  • Enables adaptive computation at inference

1. Specialized Architectures:

• • MiniCPM reranker: Compact model optimized for reranking
  • Custom attention patterns for cross-encoder efficiency
  • LoRA adaptation for parameter-efficient tuning

1. Inference:

• • Score candidates: s_i = Reranker(query, doc_i)
  • Rerank: docs_sorted = sort(docs, key=scores, descending=True)
  • Return top-k after reranking

1. Evaluation:

• • Metrics: MRR@10, nDCG@10, MAP
  • Benchmarks: MSMARCO, BEIR reranking tasks
  • Measure latency vs. quality trade-offs

The key advantage over embedding models is that rerankers see the full query-document interaction, enabling more nuanced relevance judgments at the cost of higher computational requirements.

Related Pages

• Implementation:FlagOpen_FlagEmbedding_LLM_Reranker_Evaluate
• Implementation:FlagOpen_FlagEmbedding_LLM_Reranker_Instruction_Data
• Implementation:FlagOpen_FlagEmbedding_LLM_Reranker_Instruction_Modeling
• Implementation:FlagOpen_FlagEmbedding_MiniCPM_Reranker_Config
• Implementation:FlagOpen_FlagEmbedding_MiniCPM_Reranker_Layerwise
• Implementation:FlagOpen_FlagEmbedding_MiniCPM_Reranker_Merge
• Implementation:FlagOpen_FlagEmbedding_LLM_Reranker_Layerwise_Modeling