Implementation:BerriAI Litellm Policy Templates

Overview

Description

The policy_templates.json file at the repository root is the authoritative source of guardrail policy templates for the LiteLLM Proxy. Policy templates are pre-configured bundles of guardrails that address common security, privacy, and compliance requirements. They enable administrators to deploy comprehensive guardrail policies with a single click through the LiteLLM Proxy UI.

Each template contains:

• Metadata for UI presentation (title, description, icon, complexity rating)
• Guardrail definitions with complete LiteLLM configuration (patterns, actions, modes)
• Template data used to instantiate the policy when a user applies it

The file is hosted on GitHub and fetched remotely by the proxy server at the /policy/templates API endpoint. A local backup copy is maintained at litellm/policy_templates_backup.json for offline fallback.

Usage

The file is fetched remotely by the proxy server's policy templates endpoint:

# Remote fetch URL:
https://raw.githubusercontent.com/BerriAI/litellm/main/policy_templates.json

The GET /policy/templates endpoint in litellm/proxy/management_endpoints/policy_endpoints.py attempts to fetch from GitHub first, falling back to the local backup on failure. The response is consumed by the LiteLLM Proxy UI to display a template selection grid.

Data Schema

Top-Level Structure

The file is a JSON array of policy template objects. Each object represents one deployable policy template.

[
    {
        "id": "template-identifier",
        "title": "Human-Readable Title",
        "description": "Detailed description of the template's purpose...",
        "icon": "ShieldCheckIcon",
        "iconColor": "text-purple-500",
        "iconBg": "bg-purple-50",
        "guardrails": ["guardrail-name-1", "guardrail-name-2", ...],
        "complexity": "High",
        "guardrailDefinitions": [
            {
                "guardrail_name": "guardrail-name-1",
                "litellm_params": { ... },
                "guardrail_info": { "description": "..." }
            },
            ...
        ],
        "templateData": {
            "policy_name": "policy-slug",
            "description": "Policy description",
            "guardrails_add": ["guardrail-name-1", ...],
            "guardrails_remove": []
        }
    },
    ...
]

Schema Fields

Policy Template Object

Field	Type	Required	Description
id	String	Yes	Unique slug identifier for the template
title	String	Yes	Display title shown in the UI template grid
description	String	Yes	Full description of the template's protection scope
icon	String	Yes	React icon component name (e.g., "ShieldCheckIcon", "ShieldExclamationIcon")
iconColor	String	Yes	Tailwind CSS text color class (e.g., "text-purple-500", "text-blue-500", "text-red-500")
iconBg	String	Yes	Tailwind CSS background color class (e.g., "bg-purple-50")
guardrails	Array[String]	Yes	Ordered list of guardrail identifiers included in this template
complexity	String	Yes	Complexity rating: "Low", "Medium", or "High"
guardrailDefinitions	Array[Object]	Yes	Complete guardrail configuration objects
templateData	Object	Yes	Data payload used when applying the template to create a policy

Guardrail Definition Object

Field	Type	Description
guardrail_name	String	Unique identifier for the guardrail
litellm_params	Object	LiteLLM runtime configuration
guardrail_info	Object	Contains a description field with human-readable explanation

litellm_params Object (Pattern-Based Guardrails)

Field	Type	Description
guardrail	String	Engine identifier, always "litellm_content_filter"
mode	String	Execution phase: "pre_call" (before LLM) or "post_call" (after LLM)
patterns	Array[Object]	Pattern rules, each with pattern_type, pattern_name, and action
pattern_redaction_format	String	Format for redacted text, e.g., "[{pattern_name}_REDACTED]" or "[PASSPORT_REDACTED]"

litellm_params Object (Category-Based Guardrails)

Field	Type	Description
guardrail	String	Engine identifier: "litellm_content_filter"
mode	String	Execution phase: "pre_call" or "post_call"
categories	Array[Object]	Category-based content filter rules

Pattern Actions

Action	Behavior
"MASK"	Replaces matched content with the redaction format string, allowing the request to proceed
"BLOCK"	Rejects the entire request when a match is found

Available Templates

Template ID	Title	Complexity	Guardrails Count
advanced-au-pii-protection	Advanced PII Protection (Australia)	High	8 guardrails
baseline-pii-protection	Baseline PII Protection	Low	3 guardrails
nsfw-content-filter-australia	NSFW Content Filter (Australia)	Medium	5 guardrails

Usage Examples

How the UI Fetches Templates

The proxy UI calls the /policy/templates endpoint to populate the template selection grid:

# API endpoint definition:
@router.get("/policy/templates", tags=["policy management"],
            dependencies=[Depends(user_api_key_auth)])
async def get_policy_templates(request: Request, ...) -> list:
    """Get policy templates for the UI (pre-configured guardrail combinations)."""
    # 1. Check if local-only mode is enabled
    use_local = os.getenv("LITELLM_LOCAL_POLICY_TEMPLATES", "").strip().lower() in (
        "true", "1", "yes"
    )
    if use_local:
        return _load_policy_templates_from_local_backup()

    # 2. Try fetching from GitHub
    try:
        response = await async_client.get(POLICY_TEMPLATES_GITHUB_URL)
        if response.status_code == 200:
            return response.json()
    except Exception:
        pass

    # 3. Fall back to local backup
    return _load_policy_templates_from_local_backup()

How a Template is Applied

When a user selects a template in the UI, the templateData object is sent to the policy creation endpoint. The guardrails_add list tells the system which guardrails to activate, and the guardrailDefinitions provide the full configuration for each guardrail:

# Example templateData sent when applying a template:
{
    "policy_name": "advanced-pii-protection-australia",
    "description": "Comprehensive PII detection and masking policy for Australia...",
    "guardrails_add": [
        "au-pii-tax-identifiers",
        "au-pii-passports",
        "international-pii-identifiers",
        "contact-information-pii",
        "financial-pii",
        "credentials-api-keys",
        "network-infrastructure-pii",
        "protected-class-information"
    ],
    "guardrails_remove": []
}

Related Pages

• Policy Templates Backup - Local backup copy bundled in the package
• Callback Configs - Related integration callback configuration