Implementation:Norrrrrrr lyn WAInjectBench process folder

Knowledge Sources	WAInjectBench
Domains	Computer_Vision, Security, Evaluation
Last Updated	2026-02-14 16:00 GMT

Overview

Concrete tool for evaluating an image detector against a single scenario folder and computing TPR/FPR, provided by the WAInjectBench main_image module.

Description

The process_folder function in main_image.py handles both standard detectors and LLaVA variants. For LLaVA detectors, it explicitly imports and calls llava.detect(folder_path, detector_name=detector_name). For all other detectors, it calls detector.detect(str(folder_path)). Detection results (file IDs as strings) are converted to integers. The total file count is obtained via folder_path.glob("*").

Usage

Called iteratively from run_experiment for every subfolder in the benign/ and malicious/ directories.

Code Reference

Source Location

• Repository: WAInjectBench
• File: main_image.py (L30-52)

Signature

def process_folder(folder_path: Path, detector, detector_name: str, is_malicious: bool) -> Dict:
    data_name = folder_path.name

    if detector_name in ["llava-1.5-7b-prompt", "llava-1.5-7b-ft"]:
        from detector_image import llava
        detect_files = llava.detect(str(folder_path), detector_name=detector_name)
    else:
        detect_files = detector.detect(str(folder_path))

    detect_ids = [int(f) for f in detect_files]
    total_num = len(list(folder_path.glob("*")))

    if is_malicious:
        rate_key, rate_value = "tpr", round(len(detect_ids) / total_num, 4) if total_num > 0 else 0.0
    else:
        rate_key, rate_value = "fpr", round(len(detect_ids) / total_num, 4) if total_num > 0 else 0.0

    return {
        "data_name": data_name,
        rate_key: rate_value,
        "detect_ids": detect_ids,
        "total_num": total_num,
    }

Import

from main_image import process_folder

I/O Contract

Inputs

Name	Type	Required	Description
folder_path	Path	Yes	Path to an image scenario folder
detector	module	Yes	Loaded detector module with detect(folder_path) -> List[int]
detector_name	str	Yes	Detector identifier string (used for LLaVA routing)
is_malicious	bool	Yes	Whether the folder is from the malicious/ directory

Outputs

Name	Type	Description
result	Dict	Contains data_name (str), tpr or fpr (float), detect_ids (List[int]), total_num (int)

Usage Examples

Evaluating a Single Folder

from pathlib import Path
from main_image import load_detector, process_folder

detector = load_detector("embedding-i")
result = process_folder(
    folder_path=Path("data/image/malicious/text_overlay_attack"),
    detector=detector,
    detector_name="embedding-i",
    is_malicious=True
)
print(f"Dataset: {result['data_name']}")
print(f"TPR: {result['tpr']}")
print(f"Detected {len(result['detect_ids'])} of {result['total_num']} images")

Related Pages

Implements Principle

• Principle:Norrrrrrr_lyn_WAInjectBench_Per_Folder_Image_Detection

Requires Environment

• Environment:Norrrrrrr_lyn_WAInjectBench_OpenAI_API_Credentials