Implementation:Explodinggradients Ragas HaystackLLMWrapper Class

Metadata	Value
Source	src/ragas/llms/haystack_wrapper.py (Lines 22-159)
Domains	LLM, Haystack
Last Updated	2026-02-10

Overview

Wraps Haystack LLM generator components as a Ragas BaseRagasLLM, enabling synchronous and asynchronous text generation using Haystack's OpenAI, Azure, and HuggingFace generators within the Ragas framework.

Description

HaystackLLMWrapper extends BaseRagasLLM and integrates Haystack generator components into Ragas. On initialization, it:

1. Lazy-imports the required Haystack modules (AsyncPipeline, generator classes).
2. Validates that the provided haystack_generator is one of the supported types: OpenAIGenerator, AzureOpenAIGenerator, HuggingFaceAPIGenerator, or HuggingFaceLocalGenerator.
3. Sets up an AsyncPipeline with the generator as the "llm" component for async execution.
4. Initializes the RunConfig.

The class provides:

• generate_text: Synchronous text generation. Converts the PromptValue to a string, runs the generator directly, and wraps the first reply in a LangChain LLMResult.

• agenerate_text: Asynchronous text generation. Constructs input with prompt and generation kwargs (temperature), runs the async pipeline, and wraps the result in an LLMResult.

• is_finished: Always returns True (single-shot generation).

• __repr__: Returns a descriptive string including the model name, extracted based on the generator type (model name, deployment name, or HuggingFace model ID).

Usage

Use this wrapper when you want to use Haystack-based LLM generators (OpenAI, Azure OpenAI, HuggingFace) as the LLM backend for Ragas evaluation metrics. This enables teams already using Haystack to evaluate their pipelines without switching LLM providers.

Code Reference

Source Location

Item	Detail
File	src/ragas/llms/haystack_wrapper.py
Lines	22-159
Module	ragas.llms.haystack_wrapper

Class Signature

class HaystackLLMWrapper(BaseRagasLLM):
    def __init__(
        self,
        haystack_generator: Union[
            AzureOpenAIGenerator,
            HuggingFaceAPIGenerator,
            HuggingFaceLocalGenerator,
            OpenAIGenerator,
        ],
        run_config: Optional[RunConfig] = None,
        cache: Optional[CacheInterface] = None,
    ) -> None: ...

    def generate_text(
        self,
        prompt: PromptValue,
        n: int = 1,
        temperature: float = 0.01,
        stop: Optional[List[str]] = None,
        callbacks: Optional[Callbacks] = None,
    ) -> LLMResult: ...

    async def agenerate_text(
        self,
        prompt: PromptValue,
        n: int = 1,
        temperature: Optional[float] = None,
        stop: Optional[List[str]] = None,
        callbacks: Optional[Callbacks] = None,
    ) -> LLMResult: ...

Import

from ragas.llms.haystack_wrapper import HaystackLLMWrapper

I/O Contract

Constructor

Name	Type	Required	Description
haystack_generator	Union[AzureOpenAIGenerator, HuggingFaceAPIGenerator, HuggingFaceLocalGenerator, OpenAIGenerator]	Yes	A Haystack generator instance
run_config	Optional[RunConfig]	No	Execution configuration (defaults to RunConfig())
cache	Optional[CacheInterface]	No	Cache backend for LLM responses

generate_text

Direction	Name	Type	Description
Input	prompt	PromptValue	The prompt to send to the generator
Input	n	int	Number of generations (default: 1)
Input	temperature	float	Sampling temperature (default: 0.01)
Input	stop	Optional[List[str]]	Stop sequences (optional)
Input	callbacks	Optional[Callbacks]	LangChain callbacks (optional)
Output	(return)	LLMResult	LangChain LLMResult containing the generated text

agenerate_text

Direction	Name	Type	Description
Input	prompt	PromptValue	The prompt to send to the generator
Input	temperature	Optional[float]	Sampling temperature (optional)
Output	(return)	LLMResult	LangChain LLMResult containing the generated text

Exceptions

Exception	Condition
ImportError	Haystack (haystack-ai) is not installed
TypeError	Generator is not one of the four supported Haystack generator types

Usage Examples

Using with OpenAI Generator

from haystack.components.generators.openai import OpenAIGenerator
from ragas.llms.haystack_wrapper import HaystackLLMWrapper

# Create a Haystack generator
generator = OpenAIGenerator(model="gpt-4o")

# Wrap it for Ragas
llm = HaystackLLMWrapper(haystack_generator=generator)

# Use with Ragas metrics
from ragas.metrics import faithfulness
faithfulness.llm = llm

Using with Azure OpenAI Generator

from haystack.components.generators.azure import AzureOpenAIGenerator
from ragas.llms.haystack_wrapper import HaystackLLMWrapper

generator = AzureOpenAIGenerator(
    azure_deployment="my-gpt-4-deployment",
    azure_endpoint="https://my-resource.openai.azure.com/",
)

llm = HaystackLLMWrapper(haystack_generator=generator)
print(llm)  # HaystackLLMWrapper(llm=my-gpt-4-deployment(...))

Async Generation

import asyncio
from langchain_core.prompt_values import StringPromptValue

prompt = StringPromptValue(text="Explain RAG in one sentence.")
result = asyncio.run(llm.agenerate_text(prompt=prompt, temperature=0.7))
print(result.generations[0][0].text)

Related Pages

• LiteLLMStructuredLLM Class - Alternative LLM wrapper using LiteLLM for structured outputs across 100+ providers
• LlamaIndex Integration - Uses LlamaIndexLLMWrapper for a similar wrapping pattern
• EvaluatorChain Class - Uses LangchainLLMWrapper for LangChain LLM integration