Principle:Promptfoo Promptfoo Provider Resolution
| Knowledge Sources | |
|---|---|
| Domains | Provider_Management, Evaluation |
| Last Updated | 2026-02-14 08:00 GMT |
Overview
A factory-based resolution mechanism that maps provider identifier strings to instantiated LLM provider objects capable of making API calls.
Description
Provider Resolution is the process of converting a provider identifier string (such as `'openai:gpt-4'` or `'python:my_provider.py'`) into a fully initialized ApiProvider object. This is essential because Promptfoo supports 70+ LLM providers and custom integrations, each requiring different SDKs, authentication, and configuration.
The resolution pattern uses a registry of factory functions, each with a `test()` predicate to match provider path prefixes and a `create()` method to instantiate the provider. This allows extensibility without modifying core logic.
The mechanism handles:
- Protocol-based routing: `file://`, `python:`, `https://`, `exec:`, `golang:`, `ruby:` prefixes
- Cloud provider resolution: Fetch provider configs from Promptfoo Cloud
- YAML/JSON file loading: Load provider definitions from config files
- Environment variable injection: Merge env overrides at multiple levels
Usage
Use this principle whenever you need to convert a provider specification from a config file into a callable provider object. This is the second step in the evaluation pipeline, after configuration loading and before test execution.
Theoretical Basis
The provider resolution algorithm follows a chain-of-responsibility pattern:
Pseudo-code Logic:
1. Render Nunjucks templates in provider path (env var substitution)
2. Check if path is a cloud provider → fetch from cloud, recurse
3. Check if path is file:// with .yaml/.yml/.json → load file, recurse
4. Iterate through providerMap registry:
a. If factory.test(path) matches → factory.create(path, options)
b. Apply transforms, delays, labels from options
5. If no factory matches → throw error with helpful message