Implementation:FlagOpen FlagEmbedding Add Reranker Score Script

Template:Metadata

Overview

API documentation for scripts/add_reranker_score.py, which scores query-passage pairs using a reranker model for knowledge distillation.

CLI Signature

python scripts/add_reranker_score.py \
    --input_file train_data_minedHN.jsonl \
    --output_file train_data_scored.jsonl \
    --reranker_name_or_path BAAI/bge-reranker-v2-m3 \
    --devices cuda:0 cuda:1

Key Dataclasses

ScoreArgs

input_file - Path to input JSONL file

output_file - Path to output JSONL file

ModelArgs

reranker_name_or_path - Name or path of the reranker model

reranker_model_class - Class of the reranker model

use_fp16 - Enable FP16 inference

use_bf16 - Enable BF16 inference

devices - List of CUDA devices to use

trust_remote_code - Whether to trust remote code

reranker_peft_path - Path to PEFT adapter for reranker

query_instruction_for_rerank - Instruction prepended to queries

passage_instruction_for_rerank - Instruction prepended to passages

normalize - Whether to normalize scores

prompt - Prompt for the reranker

cutoff_layers - Cutoff layers for layerwise reranker

compress_layers - Compress layers configuration

compress_ratio - Compress ratio configuration

reranker_batch_size - Batch size for reranker inference

reranker_max_length - Maximum input length for reranker

I/O

Input

JSONL file with query, pos (list of positive passages), and neg (list of negative passages) fields.

Output

JSONL file with the same fields plus added pos_scores: List[float] and neg_scores: List[float].

Internal Logic

1. Loads the reranker via FlagAutoReranker.from_finetuned()
2. Creates all (query, passage) pairs from both positive and negative passages
3. Calls compute_score() on all pairs
4. Assigns the resulting scores back to pos_scores and neg_scores in the output JSONL

Related Pages

Principle:FlagOpen_FlagEmbedding_Knowledge_Distillation_Scoring