Implementation:Vibrantlabsai Ragas HaystackLLMWrapper

Knowledge Sources	Vibrantlabsai_Ragas
Domains	LLM Integration, Haystack, Evaluation
Last Updated	2026-02-12 00:00 GMT

Overview

HaystackLLMWrapper is a wrapper class that integrates Haystack LLM generator components into the Ragas evaluation framework, enabling both synchronous and asynchronous text generation.

Description

HaystackLLMWrapper extends BaseRagasLLM to bridge between Haystack's generator ecosystem and the Ragas evaluation toolkit. It accepts any supported Haystack generator (OpenAIGenerator, AzureOpenAIGenerator, HuggingFaceAPIGenerator, or HuggingFaceLocalGenerator) and wraps it with an interface compatible with Ragas metric computation. The class performs lazy imports of Haystack dependencies, raising a helpful error if Haystack is not installed. It constructs an AsyncPipeline internally to support asynchronous generation via the agenerate_text method. Both synchronous and asynchronous generation methods return LLMResult objects containing Generation instances, conforming to the LangChain output format used internally by Ragas.

Usage

Use this class when evaluating LLM applications built with the Haystack framework. It allows Ragas metrics to call Haystack generators directly, avoiding the need to re-implement generation logic or switch frameworks during evaluation.

Code Reference

Source Location

• Repository: Vibrantlabsai_Ragas
• File: src/ragas/llms/haystack_wrapper.py

Signature

class HaystackLLMWrapper(BaseRagasLLM):
    def __init__(
        self,
        haystack_generator: t.Union[
            "AzureOpenAIGenerator",
            "HuggingFaceAPIGenerator",
            "HuggingFaceLocalGenerator",
            "OpenAIGenerator",
        ],
        run_config: t.Optional[RunConfig] = None,
        cache: t.Optional[CacheInterface] = None,
    ):

Import

from ragas.llms.haystack_wrapper import HaystackLLMWrapper

I/O Contract

Inputs

Name	Type	Required	Description
haystack_generator	Union[AzureOpenAIGenerator, HuggingFaceAPIGenerator, HuggingFaceLocalGenerator, OpenAIGenerator]	Yes	An instance of a supported Haystack generator component
run_config	RunConfig	No	Configuration object to manage LLM execution settings; defaults to a new RunConfig if not provided
cache	CacheInterface	No	A cache instance for storing and retrieving previous results

Outputs

Name	Type	Description
generate_text return	LLMResult	Synchronous generation result containing a list of Generation objects with the generated text
agenerate_text return	LLMResult	Asynchronous generation result containing a list of Generation objects with the generated text

Key Methods

Method	Description
generate_text(prompt, n, temperature, stop, callbacks)	Synchronous text generation using the Haystack generator's run method
agenerate_text(prompt, n, temperature, stop, callbacks)	Asynchronous text generation using the Haystack AsyncPipeline's run_async method
is_finished(response)	Always returns True, indicating the response is complete

Usage Examples

Basic Usage

from haystack.components.generators.openai import OpenAIGenerator
from ragas.llms.haystack_wrapper import HaystackLLMWrapper

# Create a Haystack OpenAI generator
haystack_gen = OpenAIGenerator(model="gpt-4")

# Wrap it for use with Ragas
llm = HaystackLLMWrapper(haystack_generator=haystack_gen)

# Use in Ragas evaluation
# The wrapper can now be passed to any Ragas metric that requires a BaseRagasLLM

Async Usage

from haystack.components.generators.openai import OpenAIGenerator
from ragas.llms.haystack_wrapper import HaystackLLMWrapper
from langchain_core.prompt_values import StringPromptValue

haystack_gen = OpenAIGenerator(model="gpt-4")
llm = HaystackLLMWrapper(haystack_generator=haystack_gen)

# Async generation
prompt = StringPromptValue(text="What is the capital of France?")
result = await llm.agenerate_text(prompt, temperature=0.5)
print(result.generations[0][0].text)

Related Pages

• Environment:Vibrantlabsai_Ragas_Python_3_9_Core_Environment