Implementation:Explodinggradients Ragas PromptMixin Save Load

PromptMixin Save/Load

PromptMixin implements the Prompt Persistence principle in the Ragas evaluation toolkit. It is a mixin class that provides prompt management capabilities (get, set, save, load, adapt) to any class that uses PydanticPrompt instances, including metrics and synthesizers.

Source Location

• File: src/ragas/prompt/mixin.py
• Class definition: Lines 17-131
• get_prompts(): Lines 32-39
• set_prompts(): Lines 41-61
• save_prompts(): Lines 86-103
• load_prompts(): Lines 105-131

Import

from ragas.prompt import PromptMixin

Or directly:

from ragas.prompt.mixin import PromptMixin

Class Definition

class PromptMixin:
    """
    Mixin class for classes that have prompts.
    eg: BaseSynthesizer, MetricWithLLM
    """
    name: str = ""

The name attribute is used as a prefix in prompt filenames. When empty, prompts are saved without a metric prefix.

Methods

get_prompts() (Lines 32-39)

def get_prompts(self) -> Dict[str, PydanticPrompt]

Returns a dictionary mapping prompt names (the PydanticPrompt.name attribute) to their PydanticPrompt instances. Internally discovers prompts by inspecting the object's attributes using inspect.getmembers() and filtering for PydanticPrompt instances.

Example:

prompts = my_metric.get_prompts()
for name, prompt in prompts.items():
    print(f"Prompt '{name}': {prompt.instruction[:80]}...")

set_prompts() (Lines 41-61)

def set_prompts(self, **prompts) -> None

Sets one or more prompts by name using keyword arguments. Each key must be a prompt name returned by get_prompts(), and each value must be a PydanticPrompt instance.

Raises:

• ValueError if a prompt name does not exist on the metric.
• ValueError if a value is not a PydanticPrompt instance.

Example:

prompts = my_metric.get_prompts()
prompts["my_prompt"].instruction = "New optimized instruction"
my_metric.set_prompts(**prompts)

save_prompts() (Lines 86-103)

def save_prompts(self, path: str) -> None

Serializes all prompts to individual JSON files in the specified directory.

Parameters:

Parameter	Type	Description
path	str	Directory path where prompt JSON files will be saved. Must already exist.

File naming convention:

• If self.name is set: {self.name}_{prompt_name}_{language}.json
• If self.name is empty: {prompt_name}_{language}.json

Raises:

• ValueError if the path does not exist.

Example:

import os
os.makedirs("./prompts", exist_ok=True)
my_metric.save_prompts("./prompts")
# Creates files like: ./prompts/my_metric_scoring_prompt_english.json

load_prompts() (Lines 105-131)

def load_prompts(self, path: str, language: Optional[str] = None) -> Dict[str, PydanticPrompt]

Loads prompts from JSON files in the specified directory and returns them as a dictionary.

Parameters:

Parameter	Type	Default	Description
path	str	required	Directory path containing prompt JSON files.
language	Optional[str]	None	Language tag for the prompt files. Defaults to "english" if not specified.

Return Value:

Returns Dict[str, PydanticPrompt] mapping prompt names to loaded prompt instances.

Raises:

• ValueError if the path does not exist.

Example:

loaded_prompts = my_metric.load_prompts("./prompts", language="english")
my_metric.set_prompts(**loaded_prompts)

adapt_prompts() (Lines 63-84)

async def adapt_prompts(
    self,
    language: str,
    llm: Union[BaseRagasLLM, InstructorBaseRagasLLM],
    adapt_instruction: bool = False,
) -> Dict[str, PydanticPrompt]

Adapts all prompts to a target language using an LLM. Returns adapted prompts that can be saved via save_prompts() and later loaded with the language tag.

Internal Discovery: _get_prompts() (Lines 25-30)

def _get_prompts(self) -> Dict[str, PydanticPrompt]

Private method that discovers prompts by inspecting the object with inspect.getmembers() and filtering for PydanticPrompt instances. Returns a dictionary mapping attribute names (not prompt names) to prompt objects. This is used internally by set_prompts() to map between prompt names and the actual attribute names on the object.

Complete Workflow Example

import os
from ragas.optimizers import GeneticOptimizer
from ragas.losses import BinaryMetricLoss

# Step 1: Optimize prompts
optimizer = GeneticOptimizer(metric=my_metric, llm=my_llm)
best_prompts = optimizer.optimize(
    dataset=annotations,
    loss=BinaryMetricLoss(metric="accuracy"),
    config={"population_size": 3},
)

# Step 2: Apply optimized instructions to the metric
prompts = my_metric.get_prompts()
for name, instruction in best_prompts.items():
    prompts[name].instruction = instruction
my_metric.set_prompts(**prompts)

# Step 3: Save to disk
os.makedirs("./optimized_prompts", exist_ok=True)
my_metric.save_prompts("./optimized_prompts")

# Step 4: In a new session, load and apply
loaded = my_metric.load_prompts("./optimized_prompts", language="english")
my_metric.set_prompts(**loaded)

Implements

• Principle:Explodinggradients_Ragas_Prompt_Persistence

See Also

• GeneticOptimizer Class -- Produces optimized prompts to persist.
• DSPyOptimizer Class -- Alternative optimizer whose output should be persisted.
• MetricAnnotation Class -- Annotation data used during optimization.