Implementation:Explodinggradients Ragas Amazon Bedrock Integration

Metadata	Value
Source	src/ragas/integrations/amazon_bedrock.py (Lines 7-135)
Domains	Integration, Amazon_Bedrock
Last Updated	2026-02-10

Overview

Provides utility functions to convert Amazon Bedrock agent trace data into Ragas message and evaluation formats, enabling Ragas-based evaluation of Bedrock agent interactions including knowledge base lookups.

Description

This module contains four functions that process Amazon Bedrock agent traces:

• get_last_orchestration_value iterates through a list of trace dictionaries to find the last occurrence of a specified key within the orchestrationTrace. It returns a tuple of the index and the value found, or (-1, None) if not found.

• extract_messages_from_model_invocation parses the JSON text field of a modelInvocationInput and converts each message into either a HumanMessage (for role "user") or AIMessage (for role "assistant"). The last message is excluded from the result (it is typically the prompt being processed).

• convert_to_ragas_messages combines the above functions: it extracts the conversation history from the last model invocation and appends the final response from the observation trace (if present after the model invocation) as an AIMessage.

• extract_kb_trace processes traces to extract knowledge base interaction groups. Each group follows a specific ordering: a knowledge base invocation input, then a knowledge base lookup output with retrieved references, and finally a final response. The function supports multiple concurrent knowledge base invocation groups and returns a list of dictionaries with keys user_input, retrieved_contexts, and response.

Usage

Use this integration when evaluating Amazon Bedrock agent applications with Ragas. It is particularly useful for:

• Converting Bedrock agent traces into Ragas message format for multi-turn evaluation.
• Extracting knowledge base retrieval contexts and responses for RAG evaluation metrics such as faithfulness and context precision.

Code Reference

Source Location

Item	Detail
File	src/ragas/integrations/amazon_bedrock.py
Lines	7-135
Module	ragas.integrations.amazon_bedrock

Signatures

def get_last_orchestration_value(traces: List[Dict[str, Any]], key: str) -> Tuple[int, Any]

def extract_messages_from_model_invocation(model_inv: Dict) -> List[Union[HumanMessage, AIMessage]]

def convert_to_ragas_messages(traces: List) -> List[Union[HumanMessage, AIMessage]]

def extract_kb_trace(traces: List) -> List[Dict[str, Any]]

Import

from ragas.integrations.amazon_bedrock import convert_to_ragas_messages, extract_kb_trace

I/O Contract

convert_to_ragas_messages

Direction	Name	Type	Description
Input	traces	List[Dict[str, Any]]	List of Bedrock agent trace dictionaries containing orchestrationTrace data
Output	(return)	List[Union[HumanMessage, AIMessage]]	Ordered list of Ragas message objects representing the conversation

extract_kb_trace

Direction	Name	Type	Description
Input	traces	List[Dict[str, Any]]	List of Bedrock agent trace dictionaries
Output	(return)	List[Dict[str, Any]]	List of dicts with keys: user_input (str), retrieved_contexts (List[str]), response (str)

get_last_orchestration_value

Direction	Name	Type	Description
Input	traces	List[Dict[str, Any]]	List of Bedrock agent trace dictionaries
Input	key	str	Key to search for within orchestrationTrace
Output	(return)	Tuple[int, Any]	Tuple of (last_index, value) or (-1, None) if not found

extract_messages_from_model_invocation

Direction	Name	Type	Description
Input	model_inv	Dict	A modelInvocationInput dictionary containing a JSON text field
Output	(return)	List[Union[HumanMessage, AIMessage]]	Extracted messages (excluding the last message)

Usage Examples

Converting Agent Traces to Ragas Messages

from ragas.integrations.amazon_bedrock import convert_to_ragas_messages

# Example Bedrock agent traces from an invoke_agent response
traces = [
    {
        "trace": {
            "orchestrationTrace": {
                "modelInvocationInput": {
                    "text": '{"messages": [{"role": "user", "content": "What is RAG?"}]}'
                }
            }
        }
    },
    {
        "trace": {
            "orchestrationTrace": {
                "observation": {
                    "finalResponse": {
                        "text": "RAG stands for Retrieval Augmented Generation."
                    }
                }
            }
        }
    },
]

messages = convert_to_ragas_messages(traces)
# Returns: [HumanMessage(content="What is RAG?"), AIMessage(content="RAG stands for...")]

Extracting Knowledge Base Traces for RAG Evaluation

from ragas.integrations.amazon_bedrock import extract_kb_trace

# Traces containing knowledge base lookup interactions
traces = [
    {
        "trace": {
            "orchestrationTrace": {
                "invocationInput": {
                    "invocationType": "KNOWLEDGE_BASE",
                    "knowledgeBaseLookupInput": {"text": "What is LLM evaluation?"},
                }
            }
        }
    },
    {
        "trace": {
            "orchestrationTrace": {
                "observation": {
                    "knowledgeBaseLookupOutput": {
                        "retrievedReferences": [
                            {"content": {"text": "LLM evaluation measures quality..."}}
                        ]
                    }
                }
            }
        }
    },
    {
        "trace": {
            "orchestrationTrace": {
                "observation": {
                    "finalResponse": {"text": "LLM evaluation is the process of..."}
                }
            }
        }
    },
]

kb_results = extract_kb_trace(traces)
# Returns: [{"user_input": "What is LLM evaluation?",
#            "retrieved_contexts": ["LLM evaluation measures quality..."],
#            "response": "LLM evaluation is the process of..."}]

Related Pages

• Messages Module - Defines the HumanMessage and AIMessage types used by this integration
• R2R Integration - Similar pattern for converting retrieval responses to Ragas datasets
• LlamaIndex Integration - Another integration that converts agent traces to Ragas messages