Heuristic:Truera Trulens Feedback Score Thresholds

Knowledge Sources	TruLens Dashboard Styles TruLens Guardrails Internal engineering practice
Domains	Evaluation, Decision_Making
Last Updated	2026-02-14 08:00 GMT

Overview

Score categorization framework using 0.6/0.8 thresholds (higher-is-better) or 0.2/0.4 thresholds (lower-is-better) to classify feedback results as PASS, WARNING, or FAIL.

Description

TruLens normalizes all feedback scores to a 0-1 range and categorizes them into three tiers for dashboard display and guardrail decisions. For higher-is-better metrics (relevance, groundedness): scores >= 0.8 are PASS, 0.6-0.8 are WARNING, < 0.6 are FAIL. For lower-is-better metrics (toxicity, bias): scores <= 0.2 are PASS, 0.2-0.4 are WARNING, > 0.4 are FAIL. Guardrail decorators (`context_filter`, `block_input`, `block_output`) use a configurable `threshold` parameter (default 0.5) for binary accept/reject decisions.

Usage

Apply this heuristic when configuring guardrail thresholds or interpreting dashboard results. The default threshold of 0.5 for guardrails is a neutral midpoint. Adjust based on your tolerance: stricter thresholds (0.7-0.8) for safety-critical applications, looser thresholds (0.3-0.4) for exploratory use.

The Insight (Rule of Thumb)

Action: Use default thresholds for initial evaluation; tune based on observed score distributions.
Values:
- Dashboard categorization: FAIL < 0.6 < WARNING < 0.8 < PASS (higher-is-better)
- Dashboard categorization: PASS < 0.2 < WARNING < 0.4 < FAIL (lower-is-better)
- Guardrail threshold: 0.5 (default binary cutoff)
- Ground truth threshold: 0.5 (default for precision/recall/F1)
Trade-off: Stricter thresholds increase safety but may reject valid inputs/outputs. Looser thresholds increase throughput but reduce quality filtering.

Reasoning

The 0.6/0.8 thresholds for higher-is-better metrics follow a common grading convention where 80%+ is "good" and 60%+ is "acceptable." The guardrail default of 0.5 represents maximum uncertainty — the point where the model is equally likely to be correct or incorrect. For production deployments, the threshold should be calibrated against your specific score distributions and risk tolerance.

The benchmark framework uses a three-tier categorization for expected scores: low (<=0.3), medium (0.3-0.7), high (>0.7), which aligns with the dashboard categories.

Code Evidence

Dashboard score categories from `src/dashboard/trulens/dashboard/ux/styles.py:41-50`:

directions = [
    FeedbackDirection("HIGHER_IS_BETTER", False, [0, 0.6, 0.8]),
    FeedbackDirection("LOWER_IS_BETTER", True, [0.2, 0.4, 1]),
]

styling = {
    "PASS": dict(color="#aaffaa70", icon="✅"),
    "WARNING": dict(color="#ffffaa70", icon="⚠️"),
    "FAIL": dict(color="#ffaaaa70", icon="🛑"),
}

Score normalization from `src/feedback/trulens/feedback/llm_provider.py:228-229`:

normalized_score = (raw_score - min_score_val) / (
    max_score_val - min_score_val
)

Default ground truth threshold from `src/feedback/trulens/feedback/groundtruth.py:879`:

threshold=0.5

Benchmark categorization from `src/benchmark/trulens/benchmark/benchmark_frameworks/experiments/dataset_preprocessing.py:117-119`:

# expected_score <= 0.3  → category 1 (low)
# 0.3 < expected_score <= 0.7 → category 2 (medium)
# expected_score > 0.7  → category 3 (high)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment