Implementation:Run llama Llama index LLMRerank

Overview

The LLMRerank module implements an LLM-based node reranking postprocessor that uses a language model to evaluate and reorder retrieved nodes by relevance. It processes nodes in configurable batches, asks the LLM to select the most relevant nodes for a query, and returns the top-N results sorted by relevance score. This module is located at llama-index-core/llama_index/core/postprocessor/llm_rerank.py (111 lines).

Purpose

This postprocessor improves retrieval quality by using an LLM as a reranker. After initial retrieval returns candidate nodes, LLMRerank presents each batch of candidates to the LLM along with the query and asks it to select the most relevant ones with relevance scores. This two-stage retrieval pattern (retrieve then rerank) typically yields better precision than relying solely on embedding similarity.

Key Components

Class: LLMRerank

A Pydantic-based node postprocessor extending BaseNodePostprocessor.

Fields

Field	Type	Default	Description
top_n	int	10	Maximum number of nodes to return after reranking.
choice_select_prompt	SerializeAsAny[BasePromptTemplate]	DEFAULT_CHOICE_SELECT_PROMPT	The prompt template used to ask the LLM to select relevant nodes.
choice_batch_size	int	10	Number of nodes to present to the LLM in each batch.
llm	LLM	Settings.llm	The language model used for reranking.

Private Attributes

Attribute	Type	Default	Description
_format_node_batch_fn	Callable	default_format_node_batch_fn	Function that formats a batch of nodes into a string for the LLM prompt.
_parse_choice_select_answer_fn	Callable	default_parse_choice_select_answer_fn	Function that parses the LLM's response into selected choices and relevance scores.

Constructor

def __init__(
    self,
    llm: Optional[LLM] = None,
    choice_select_prompt: Optional[BasePromptTemplate] = None,
    choice_batch_size: int = 10,
    format_node_batch_fn: Optional[Callable] = None,
    parse_choice_select_answer_fn: Optional[Callable] = None,
    top_n: int = 10,
) -> None

If llm is not provided, it defaults to Settings.llm. If choice_select_prompt is not provided, it defaults to DEFAULT_CHOICE_SELECT_PROMPT.

Methods

Method	Description
_get_prompts	Returns a dictionary with the choice_select_prompt keyed as "choice_select_prompt".
_update_prompts	Updates the choice select prompt if the key "choice_select_prompt" is present in the provided dictionary.
class_name	Returns the string "LLMRerank".
_postprocess_nodes	Core reranking logic (detailed below).

Reranking Algorithm

The _postprocess_nodes method implements the following algorithm:

1. Validation: Raises ValueError if query_bundle is None. Returns an empty list if no nodes are provided.
2. Batch processing: Iterates over the input nodes in batches of choice_batch_size.
3. For each batch:
   • Extracts the underlying node objects from NodeWithScore wrappers.
   • Formats the batch into a text string using _format_node_batch_fn.
   • Calls self.llm.predict() with the choice select prompt, passing the formatted batch and the query string.
   • Parses the LLM's response using _parse_choice_select_answer_fn, which returns a list of choice indices and optional relevance scores.
   • Converts 1-based choice indices to 0-based and retrieves the corresponding nodes.
   • Creates NodeWithScore objects with the relevance scores (defaulting to 1.0 if no scores are returned).
   • Appends all selected nodes to the accumulator.
4. Final sorting: Sorts all selected nodes by score in descending order and returns the top top_n results.

Dependencies

Module	Items Imported
llama_index.core.bridge.pydantic	Field, PrivateAttr, SerializeAsAny
llama_index.core.indices.utils	default_format_node_batch_fn, default_parse_choice_select_answer_fn
llama_index.core.llms.llm	LLM
llama_index.core.postprocessor.types	BaseNodePostprocessor
llama_index.core.prompts	BasePromptTemplate
llama_index.core.prompts.default_prompts	DEFAULT_CHOICE_SELECT_PROMPT
llama_index.core.prompts.mixin	PromptDictType
llama_index.core.schema	NodeWithScore, QueryBundle
llama_index.core.settings	Settings (for default LLM)

Design Notes

• Batched processing: Nodes are processed in batches to stay within LLM context window limits. Each batch is independently evaluated by the LLM.
• Pluggable formatting and parsing: Both the batch formatting function and the answer parsing function can be replaced via constructor parameters, enabling custom prompt formats and response parsing strategies.
• Prompt management: The class integrates with LlamaIndex's prompt mixin system via _get_prompts and _update_prompts, allowing prompts to be inspected and modified after construction.
• Score handling: If the parsing function does not return relevance scores, all selected nodes default to a score of 1.0. The final sorting ensures the most relevant nodes (across all batches) are returned.
• 1-based to 0-based index conversion: The LLM produces 1-based choice indices (natural for human-readable prompts), which are converted to 0-based indices for array access.