Principle:Promptfoo Promptfoo Remote Assertion Evaluation
| Knowledge Sources | |
|---|---|
| Domains | Grading, Remote_API |
| Last Updated | 2026-02-14 07:45 GMT |
Overview
Principle governing the delegation of assertion evaluation to remote services when local grading is insufficient or when third-party scoring APIs are required.
Description
Remote Assertion Evaluation defines how promptfoo offloads assertion grading to external services. This covers two scenarios: (1) server-side evaluation via the promptfoo remote API for complex grading tasks (moderation, specialized LLM-based judging), and (2) third-party scoring via the WithPi API for custom question-based scoring specifications. Both scenarios use the cached fetch system, return standardized GradingResult objects, and handle authentication via environment variables.
Usage
Apply this principle when adding new remote assertion types or integrating additional third-party evaluation APIs.
Theoretical Basis
Remote evaluation follows the Remote Procedure Call (RPC) Pattern:
- Local grading logic serializes the evaluation context into a payload
- The payload is sent to a remote endpoint via HTTP POST
- The remote service performs the evaluation and returns a standardized result
- The local system consumes the result identically to a locally-produced GradingResult
This abstraction allows transparent switching between local and remote evaluation strategies.