Implementation:Promptfoo Promptfoo ApiProvider Interface

Knowledge Sources	Promptfoo Custom Providers
Domains	Provider_Management, API_Design
Last Updated	2026-02-14 08:00 GMT

Overview

Concrete TypeScript interface definition that all LLM providers must implement for evaluation compatibility, defined in the Promptfoo type system.

Description

The ApiProvider interface is the core abstraction in Promptfoo's provider system. It defines the minimum contract (id() and callApi) plus optional methods for embeddings, classification, sessions, and cleanup. All 70+ built-in providers and all custom providers conform to this interface.

This is a Pattern Doc documenting the interface that users must implement when creating custom providers.

Usage

Reference this interface when implementing custom JavaScript/TypeScript providers. Python and Ruby providers achieve the same contract through their respective bridge implementations.

Code Reference

Source Location

• Repository: promptfoo
• File: src/types/providers.ts
• Lines: L102-120

Signature

export interface ApiProvider {
  id: () => string;
  callApi: CallApiFunction;
  callClassificationApi?: (prompt: string) => Promise<ProviderClassificationResponse>;
  callEmbeddingApi?: (input: string) => Promise<ProviderEmbeddingResponse>;
  config?: any;
  delay?: number;
  getSessionId?: () => string;
  inputs?: Inputs;
  label?: ProviderLabel;
  transform?: string;
  toJSON?: () => any;
  cleanup?: () => void | Promise<void>;
}

// CallApiFunction signature:
type CallApiFunction = (
  prompt: string,
  context?: CallApiContextParams,
  options?: CallApiOptionsParams,
) => Promise<ProviderResponse>;

Import

import type { ApiProvider, CallApiFunction } from './types/providers';

I/O Contract

Inputs

Name	Type	Required	Description
prompt	string	Yes	The rendered prompt to send to the LLM
context	CallApiContextParams	No	Filters, vars, test metadata, trace headers
options	CallApiOptionsParams	No	Log probs flag, abort signal

Outputs

Name	Type	Description
output	string or object	The LLM's response text or structured output
tokenUsage	TokenUsage	Token counts: prompt, completion, total, cached
cost	number	Estimated cost in USD
cached	boolean	Whether the response was served from cache
error	string	Error message if the call failed

Usage Examples

Custom TypeScript Provider

// my-provider.ts
import type { ApiProvider, ProviderResponse } from 'promptfoo';

const provider: ApiProvider = {
  id: () => 'my-custom-provider',

  callApi: async (prompt: string): Promise<ProviderResponse> => {
    const response = await fetch('https://my-api.example.com/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ message: prompt }),
    });
    const data = await response.json();
    return {
      output: data.reply,
      tokenUsage: { total: data.tokens },
    };
  },
};

export default provider;

Custom Python Provider

# my_provider.py
def call_api(prompt, options, context):
    """Promptfoo calls this function for each test case."""
    # Your LLM call here
    return {
        "output": f"Response to: {prompt}",
    }

Related Pages

Implements Principle

• Principle:Promptfoo_Promptfoo_Provider_Interface