Implementation:FlagOpen FlagEmbedding AbsEvalRunner Get Models
| Type | API Doc |
|---|---|
| Source | FlagEmbedding/abc/evaluation/runner.py: L36-90
|
| Import | from FlagEmbedding.abc.evaluation.runner import AbsEvalRunner
|
Signature
@staticmethod
def get_models(model_args: AbsEvalModelArgs) -> Tuple[AbsEmbedder, Union[AbsReranker, None]]:
Parameters
| Parameter | Type | Description |
|---|---|---|
| model_args | AbsEvalModelArgs |
Dataclass containing all model configuration: embedder path, model class, reranker path, precision, batch sizes, max lengths, device config, instructions, and more. |
Returns
| Type | Description |
|---|---|
Tuple[AbsEmbedder, Union[AbsReranker, None]] |
A tuple of (embedder, reranker). The embedder is always returned. The reranker is None if model_args.reranker_name_or_path is not provided.
|
Description
This static method loads the embedding and optional reranker models from the provided model arguments. It is the core model-loading logic used by AbsEvalRunner.
Embedder Loading
The embedder is always loaded via FlagAutoModel.from_finetuned() with the following parameters extracted from model_args:
embedder = FlagAutoModel.from_finetuned(
model_name_or_path=model_args.embedder_name_or_path,
model_class=model_args.embedder_model_class,
normalize_embeddings=model_args.normalize_embeddings,
pooling_method=model_args.pooling_method,
use_fp16=model_args.use_fp16,
query_instruction_for_retrieval=model_args.query_instruction_for_retrieval,
query_instruction_format=model_args.query_instruction_format_for_retrieval,
devices=model_args.devices,
examples_for_task=model_args.examples_for_task,
examples_instruction_format=model_args.examples_instruction_format,
trust_remote_code=model_args.trust_remote_code,
cache_dir=model_args.cache_dir,
batch_size=model_args.embedder_batch_size,
query_max_length=model_args.embedder_query_max_length,
passage_max_length=model_args.embedder_passage_max_length,
)
After loading, the method explicitly sets the model's _name_or_path config attribute to model_args.embedder_name_or_path to ensure the model name is correctly tracked for output directory naming.
Reranker Loading (Optional)
If model_args.reranker_name_or_path is not None, the reranker is loaded via FlagAutoReranker.from_finetuned():
reranker = FlagAutoReranker.from_finetuned(
model_name_or_path=model_args.reranker_name_or_path,
model_class=model_args.reranker_model_class,
peft_path=model_args.reranker_peft_path,
use_fp16=model_args.use_fp16,
use_bf16=model_args.use_bf16,
query_instruction_for_rerank=model_args.query_instruction_for_rerank,
query_instruction_format=model_args.query_instruction_format_for_rerank,
passage_instruction_for_rerank=model_args.passage_instruction_for_rerank,
passage_instruction_format=model_args.passage_instruction_format_for_rerank,
cache_dir=model_args.cache_dir,
trust_remote_code=model_args.trust_remote_code,
devices=model_args.devices,
normalize=model_args.normalize,
prompt=model_args.prompt,
cutoff_layers=model_args.cutoff_layers,
compress_layers=model_args.compress_layers,
compress_ratio=model_args.compress_ratio,
batch_size=model_args.reranker_batch_size,
query_max_length=model_args.reranker_query_max_length,
max_length=model_args.reranker_max_length,
)
Similarly, the reranker's _name_or_path config attribute is set after loading.
Input / Output
Input: A single AbsEvalModelArgs dataclass instance containing all model configuration parameters.
Output: A tuple of (AbsEmbedder, Optional[AbsReranker]). The embedder is always present; the reranker is None when no reranker path is specified.
Usage Context
This method is called internally by AbsEvalRunner.load_retriever_and_reranker(), which wraps the returned models in evaluation classes:
def load_retriever_and_reranker(self):
embedder, reranker = self.get_models(self.model_args)
retriever = EvalDenseRetriever(
embedder,
search_top_k=self.eval_args.search_top_k,
overwrite=self.eval_args.overwrite
)
if reranker is not None:
reranker = EvalReranker(reranker, rerank_top_k=self.eval_args.rerank_top_k)
return retriever, reranker