Implementation:BerriAI Litellm LLM Guard

Attribute	Value
Sources	enterprise/litellm_enterprise/enterprise_callbacks/llm_guard.py
Domains	Guardrails, Content Moderation, Enterprise Callbacks
Last Updated	2026-02-15 16:00 GMT

Overview

_ENTERPRISE_LLMGuard is a content moderation integration that validates LLM prompts and streaming responses against an LLM Guard API instance, blocking requests that violate content safety policies and optionally running in key-specific or request-specific modes.

Description

The _ENTERPRISE_LLMGuard class extends CustomLogger and integrates with the LLM Guard open-source project. It sends prompts to the LLM Guard /analyze/prompt endpoint and blocks requests that are flagged as invalid.

Three Operating Modes (controlled by litellm.llm_guard_mode):

"all" -- All requests are checked.
"key-specific" -- Only requests from API keys with enable_llm_guard_check: True in their permissions are checked.
"request-specific" -- Only requests with enable_llm_guard_check: True in their metadata permissions are checked.

Supported Call Types: completion, embeddings, image_generation, moderation, audio_transcription.

The class also provides a streaming post-call hook (async_post_call_streaming_hook) that validates each streaming response chunk.

Requires the LLM_GUARD_API_BASE environment variable.

Usage

Register _ENTERPRISE_LLMGuard as a callback in the LiteLLM proxy to add LLM Guard content moderation.

Code Reference

Source Location

enterprise/litellm_enterprise/enterprise_callbacks/llm_guard.py

Signature

class _ENTERPRISE_LLMGuard(CustomLogger):
    def __init__(self, mock_testing: bool = False, mock_redacted_text: Optional[dict] = None): ...
    async def moderation_check(self, text: str): ...
    def should_proceed(self, user_api_key_dict: UserAPIKeyAuth, data: dict) -> bool: ...
    async def async_moderation_hook(
        self, data: dict, user_api_key_dict: UserAPIKeyAuth, call_type: CallTypesLiteral
    ): ...
    async def async_post_call_streaming_hook(self, user_api_key_dict: UserAPIKeyAuth, response: str): ...

Import

from litellm_enterprise.enterprise_callbacks.llm_guard import _ENTERPRISE_LLMGuard

I/O Contract

Inputs

Parameter	Type	Description
`LLM_GUARD_API_BASE` (env)	`str`	Base URL of the LLM Guard API instance (e.g., `http://localhost:8000/`).
`mock_testing`	`bool`	If `True`, skips API base validation (for testing).
`mock_redacted_text`	`Optional[dict]`	Mock response for testing.
`data`	`dict`	Request data containing messages/prompt.
`call_type`	`CallTypesLiteral`	Type of LLM call being made.

Outputs

Output	Type	Description
Pass-through	`None`	Request proceeds if LLM Guard marks it as valid.
Block	`HTTPException(400)`	Raised with `"Violated content safety policy"` when `is_valid` is `False`.
Streaming check	`str`	Returns the response string if valid; raises exception otherwise.

Usage Examples

# Proxy config for all-requests mode
litellm_settings:
  llm_guard_mode: "all"
  callbacks:
    - llm_guard

# Environment
# LLM_GUARD_API_BASE=http://localhost:8000/

# Key-specific mode: only keys with permission are checked
litellm_settings:
  llm_guard_mode: "key-specific"
  callbacks:
    - llm_guard

# API key must have: {"permissions": {"enable_llm_guard_check": true}}

Related Pages

BerriAI_Litellm_Aporia_AI_Guardrail -- Aporia AI guardrail integration
BerriAI_Litellm_Secret_Detection -- Secret detection guardrail
BerriAI_Litellm_Enterprise_Custom_Guardrail -- Tag-based guardrail mode control

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment