Principle:Promptfoo Promptfoo CSV Test Case Parsing
| Knowledge Sources | |
|---|---|
| Domains | Data_Parsing, Testing |
| Last Updated | 2026-02-14 07:45 GMT |
Overview
Principle governing the conversion of CSV-formatted tabular data into structured test cases and assertions for the evaluation pipeline.
Description
CSV Test Case Parsing defines how CSV data (from files, Google Sheets, or SharePoint) is transformed into promptfoo's internal TestCase and Assertion structures. The principle covers a domain-specific assertion string syntax (e.g., contains:value, similar(0.9):text, not-javascript:fn()) that is parsed into typed Assertion objects with proper type resolution, threshold extraction, and negation handling. This module is shared between frontend and backend to ensure consistent parsing behavior.
Usage
Apply this principle when extending the CSV input format, adding new assertion shorthand syntaxes, or modifying how test case variables are extracted from column headers.
Theoretical Basis
The parsing uses a Grammar-Based Parser with a regex-defined grammar for assertion strings:
# Pseudo-grammar for assertion strings:
# assertion := [not-] type [(threshold)] [:value]
# type := one of the BaseAssertionTypes
# threshold := float (e.g., 0.9)
# value := arbitrary string (may contain colons)
The regex is lazily compiled and cached. Special-case handling exists for legacy prefixes (javascript:, fn:, eval:, grade:).