Implementation:Run llama Llama index StructuredLLMRerank

Knowledge Sources	Run_llama_Llama_index
Domains	Postprocessing, Reranking, LLM, StructuredOutput
Last Updated	2026-02-11 19:00 GMT

Overview

StructuredLLMRerank is a node postprocessor that uses an LLM's structured prediction (function calling) capabilities to score and rerank retrieved nodes based on relevance to the query.

Description

StructuredLLMRerank extends BaseNodePostprocessor and leverages LLM structured output to obtain typed relevance scores for each retrieved document. Unlike RankGPTRerank which asks for a permutation ranking, this postprocessor asks the LLM to assign a relevance score from 1-10 to each document using a structured Pydantic output schema.

The reranking process works in batches controlled by choice_batch_size (default 10):

1. Nodes in each batch are formatted into a text representation using format_node_batch_fn (defaults to default_format_node_batch_fn).
2. The formatted batch and query are sent to the LLM via structured_predict with the DocumentRelevanceList output class.
3. The LLM returns a list of DocumentWithRelevance objects, each containing a document number and a relevance score.
4. Results across all batches are aggregated, sorted by relevance score in descending order, and the top top_n nodes are returned.

The module defines two supporting Pydantic models:

• DocumentWithRelevance - Holds a document_number and a relevance score (1-10, specified via json_schema_extra).
• DocumentRelevanceList - Contains a list of DocumentWithRelevance items.

The implementation includes robust error handling: if structured prediction fails, it can either raise an error or (when raise_on_structured_prediction_failure is False) log a warning and assign a score of 0.0 to the affected nodes. It also integrates with the instrumentation system, dispatching ReRankStartEvent and ReRankEndEvent events.

A warning is logged if the LLM does not support function calling, as this postprocessor relies on structured output.

Usage

Use StructuredLLMRerank when you want typed, scored relevance assessments from a function-calling LLM rather than a simple permutation ranking. This is particularly useful when you need numeric relevance scores for downstream filtering or when combining results from multiple retrieval sources with different score scales.

Code Reference

Source Location

• Repository: Run_llama_Llama_index
• File: llama-index-core/llama_index/core/postprocessor/structured_llm_rerank.py

Signature

class StructuredLLMRerank(BaseNodePostprocessor):
    def __init__(
        self,
        llm: Optional[LLM] = None,
        choice_select_prompt: Optional[BasePromptTemplate] = None,
        choice_batch_size: int = 10,
        format_node_batch_fn: Optional[Callable] = None,
        parse_choice_select_answer_fn: Optional[Callable] = None,
        document_relevance_list_cls: Optional[type] = None,
        raise_on_structured_prediction_failure: bool = True,
        top_n: int = 10,
    ) -> None:

Import

from llama_index.core.postprocessor.structured_llm_rerank import StructuredLLMRerank

I/O Contract

Inputs

Name	Type	Required	Description
llm	LLM	No	LLM with function calling support for structured prediction. Defaults to Settings.llm.
choice_select_prompt	BasePromptTemplate	No	Prompt template for the choice selection task. Defaults to STRUCTURED_CHOICE_SELECT_PROMPT.
choice_batch_size	int	No	Number of nodes to evaluate per LLM call. Defaults to 10.
format_node_batch_fn	Callable	No	Function to format a batch of nodes into text. Defaults to default_format_node_batch_fn.
parse_choice_select_answer_fn	Callable	No	Function to parse the structured LLM response. Defaults to default_parse_structured_choice_select_answer.
document_relevance_list_cls	type	No	Pydantic model class for structured output. Defaults to DocumentRelevanceList.
raise_on_structured_prediction_failure	bool	No	Whether to raise on prediction failure or gracefully assign score 0. Defaults to True.
top_n	int	No	Number of top-scored nodes to return. Defaults to 10.

Outputs

Name	Type	Description
nodes	List[NodeWithScore]	Top top_n nodes sorted by LLM-assigned relevance scores in descending order.

Usage Examples

from llama_index.core.postprocessor.structured_llm_rerank import StructuredLLMRerank

# Basic usage with defaults
reranker = StructuredLLMRerank(top_n=5)

query_engine = index.as_query_engine(
    node_postprocessors=[reranker]
)
response = query_engine.query("Explain quantum computing.")

# Custom LLM with graceful failure handling
from llama_index.llms.openai import OpenAI

reranker = StructuredLLMRerank(
    llm=OpenAI(model="gpt-4"),
    top_n=5,
    choice_batch_size=5,
    raise_on_structured_prediction_failure=False,
)

Related Pages

• Environment:Run_llama_Llama_index_Python_LlamaIndex_Core
• Implementation:Run_llama_Llama_index_BaseNodePostprocessor - Parent abstract base class
• Implementation:Run_llama_Llama_index_PromptMixin - Prompt management mixin used for prompt get/update