Workflow:Protectai Llm guard LLM Input Output Scanning
| Knowledge Sources | |
|---|---|
| Domains | LLM_Security, Prompt_Scanning, Output_Scanning |
| Last Updated | 2026-02-14 12:00 GMT |
Overview
End-to-end process for securing LLM interactions by scanning user prompts before they reach the model and scanning model outputs before they reach the user.
Description
This workflow implements the core LLM Guard security pipeline: a two-phase scanning process that wraps any LLM API call. In the first phase, user prompts are run through a configurable list of input scanners that detect prompt injections, toxic content, secrets, PII, and other threats. If the prompt passes validation, it is sent to the LLM. In the second phase, the model's response is run through output scanners that detect bias, hallucinations, refusals, sensitive data leakage, and malicious URLs. The pipeline returns sanitized text along with per-scanner validity flags and risk scores.
Usage
Execute this workflow when you need to integrate LLM Guard scanning into an application that calls any LLM API (OpenAI, AWS Bedrock, Google Gemini, or others). Use it whenever user-supplied prompts must be validated for security threats before reaching the model, and model outputs must be checked for policy violations before reaching the user.
Execution Steps
Step 1: Install and import LLM Guard
Install the llm-guard package and import the core scanning functions along with the specific input and output scanner classes required for your security policy. The two primary entry points are scan_prompt for input scanning and scan_output for output scanning.
Key considerations:
- Install with pip: the package name is llm-guard
- Import only the scanners you need to minimize startup time and memory usage
- If using the Anonymize/Deanonymize pair, also import and instantiate a Vault object
Step 2: Configure input scanners
Instantiate a list of input scanner objects with your desired configuration. Each scanner targets a specific threat category (e.g., prompt injection, toxicity, PII, secrets, token limits). Scanners are applied sequentially in the order they appear in the list.
Key considerations:
- Order matters: place fast, cheap scanners (InvisibleText, TokenLimit, BanSubstrings) before expensive ML-based scanners (PromptInjection, Toxicity)
- Each scanner accepts configuration parameters like thresholds, model selections, and ONNX optimization flags
- The Anonymize scanner requires a shared Vault instance for later deanonymization
Step 3: Configure output scanners
Instantiate a list of output scanner objects with your desired configuration. Output scanners validate and sanitize the model's response. They check for bias, refusals, factual consistency, sensitive data exposure, malicious URLs, and more.
Key considerations:
- The Deanonymize scanner must receive the same Vault instance used by the Anonymize input scanner
- Output scanners receive both the original prompt and the model output for context-aware analysis
- Relevance and FactualConsistency scanners compare output against the prompt to detect hallucinations
Step 4: Scan the prompt
Pass the user's prompt through the input scanner pipeline using the scan_prompt function. This returns the sanitized prompt, a dictionary of per-scanner validity flags, and a dictionary of per-scanner risk scores. Check validity results and decide whether to proceed with the LLM call or reject the prompt.
Key considerations:
- Enable fail_fast mode to stop scanning after the first failure, reducing latency for invalid prompts
- The sanitized prompt may differ from the original if scanners like Anonymize or Secrets performed redaction
- Risk scores range from 0.0 (no risk) to 1.0 (high risk)
Step 5: Call the LLM API
Send the sanitized prompt to your LLM provider (OpenAI, AWS Bedrock, Google Gemini, or any other). Use the sanitized version of the prompt rather than the original to ensure redacted content does not reach the model.
Key considerations:
- Always send the sanitized prompt, not the original user input
- For streaming responses, accumulate chunks before scanning the output
- For parallel execution, run the prompt scan concurrently with the LLM call and cancel the call if scanning fails
Step 6: Scan the output
Pass the LLM's response through the output scanner pipeline using the scan_output function. This returns the sanitized output, per-scanner validity flags, and per-scanner risk scores. Check validity results and decide whether to return the response to the user or reject it.
Key considerations:
- The output scanner requires both the sanitized prompt and the model output as inputs
- Deanonymize will restore original entities from the Vault if Anonymize was used on input
- If any scanner marks the output as invalid, the response should be blocked or flagged for review
Step 7: Return sanitized response
Return the sanitized output to the user. If all scanners passed, the response is safe to display. Log the per-scanner risk scores for monitoring and auditing purposes.
Key considerations:
- Log all scanner results (valid/invalid flags and risk scores) for observability
- Consider caching scanner results for repeated prompts to reduce latency
- Monitor scanner performance metrics to identify bottlenecks in the pipeline