Principle:Norrrrrrr lyn WAInjectBench Per File Text Detection
| Knowledge Sources | |
|---|---|
| Domains | NLP, Security, Evaluation |
| Last Updated | 2026-02-14 16:00 GMT |
Overview
An evaluation pattern that applies a text prompt injection detector to a single JSONL data file and computes the detection rate (TPR or FPR) for that file.
Description
Per-file detection is the core evaluation unit in the text detection pipeline. Given a JSONL file and a loaded detector module, the system invokes the detector's detect() function to obtain a list of flagged sample IDs, counts the total number of lines in the file, and computes either the True Positive Rate (for malicious files) or False Positive Rate (for benign files). The result is a structured dictionary containing the dataset name, detection rate, flagged IDs, and total count.
This file-level granularity enables per-scenario analysis of detector performance across different attack types or benign contexts.
Usage
Use this pattern when evaluating a detector against each individual data file in the benchmark. It is called iteratively for every .jsonl file discovered in the data directory.
Theoretical Basis
# Per-file detection evaluation pattern
detect_ids = detector.detect(file_path) # Get flagged IDs
total_num = count_lines(file_path) # Total samples
rate = len(detect_ids) / total_num # Detection rate
metric = "tpr" if is_malicious else "fpr" # Metric type depends on label
The detection rate is a simple ratio: the number of samples flagged by the detector divided by the total number of samples. For malicious data this is the True Positive Rate (recall); for benign data this is the False Positive Rate.