Principle:Liu00222 Open Prompt Injection Task Creation
| Knowledge Sources | |
|---|---|
| Domains | NLP, Data_Loading, Prompt_Injection |
| Last Updated | 2026-02-14 15:00 GMT |
Overview
A factory pattern for instantiating NLP benchmark task objects that provide iterable data samples with system prompts for prompt injection experiments.
Description
Task Creation abstracts the process of loading NLP benchmark datasets (SST-2, SMS Spam, HSOL, JFLEG, Gigaword, MRPC, RTE, Math500, Compromise) into a uniform interface. Each task object encapsulates a system prompt, dataset samples, and iteration logic. Tasks are divided into TargetTask (the legitimate task being attacked) and InjectedTask (the attacker's goal task), enabling systematic evaluation of prompt injection attacks across diverse NLP benchmarks.
Usage
Use this principle when you need to set up both the target task (what the LLM application should do) and the injected task (what the attacker wants the LLM to do instead). It is the second step in the experiment pipeline, immediately after configuration loading.
Theoretical Basis
The task creation follows a Factory Method pattern where a single function dispatches to the correct task class based on a configuration dictionary:
Pseudo-code Logic:
# Abstract factory pattern
dataset_name = config["dataset_info"]["dataset"]
if for_injection:
task = InjectedTask(config, data_num)
else:
task = TargetTask(config, data_num)
# task provides: iteration over (data_prompt, label) pairs
# task provides: get_instruction() for system prompt
The key distinction between TargetTask and InjectedTask:
- TargetTask: Uses the standard system prompt; represents the legitimate application
- InjectedTask: Uses injection-specific prompts; provides data for the attacker to inject