Workflow:Helicone Helicone Integrate Provider To Gateway
| Knowledge Sources | |
|---|---|
| Domains | LLM_Ops, AI_Gateway, Provider_Integration |
| Last Updated | 2026-02-14 06:00 GMT |
Overview
End-to-end process for adding a new LLM model provider to the Helicone AI Gateway, including provider definition, model registration, endpoint pricing configuration, and test validation.
Description
This workflow covers the complete procedure for integrating a new LLM provider into the Helicone AI Gateway. The integration involves creating a provider class that defines how to communicate with the provider API (URL construction, authentication, request/response format), defining model metadata (context length, modality, tokenizer), configuring per-endpoint pricing with tiered rate structures, registering the models in the central registry for O(1) lookups, adding the provider to the web dashboard, and writing comprehensive tests. The result is a fully functional provider that works with both Pass-Through Billing (PTB) and Bring Your Own Key (BYOK) modes, with automatic cost calculation and request routing.
The architecture uses an author/model/endpoint hierarchy: authors are companies that create models (e.g., Anthropic), models are individual model definitions (e.g., claude-3.5-sonnet), and endpoints are model-provider deployment combinations with specific pricing (e.g., claude-3.5-sonnet on Bedrock us-east-1).
Usage
Execute this workflow when you need to add support for a new LLM provider (inference host) or a new model family from an existing author. This is required whenever Helicone needs to route requests to a provider not yet in the registry, calculate costs for new models, or display new provider options in the web dashboard.
Execution Steps
Step 1: Create Provider Definition
Create a new TypeScript class extending BaseProvider in the providers directory. The class defines the provider's display name, base URL, authentication method, pricing documentation URLs, and model listing pages. For OpenAI-compatible providers, only the base URL and basic metadata need to be specified. For non-standard providers, override methods for custom body transformation, header construction, and URL building.
Key considerations:
- OpenAI-compatible providers inherit standard Bearer token authentication from BaseProvider
- Non-standard providers must override buildBody(), buildHeaders(), and buildUrl() methods
- The provider class determines how the AI Gateway constructs outbound requests to this provider
Step 2: Register Provider in Index
Add the new provider instance to the providers index file, which exports all provider instances as a single object. This makes the provider available to the routing system for request forwarding and endpoint resolution.
Key considerations:
- Provider keys must be lowercase identifiers (e.g., "deepinfra", "bedrock")
- The provider key is used throughout the system as the canonical provider identifier
Step 3: Add Provider to Web Dashboard
Register the provider in the web frontend data file with its display name, logo URL, description, documentation link, and API key label. This enables the provider to appear in the dashboard UI for key management and provider selection.
Key considerations:
- Logo assets should be placed in the web public assets directory
- The relevanceScore determines display ordering in the provider list
Step 4: Define Model Author and Metadata
Create the author directory structure with model definitions. Each model includes its human-readable name, author slug, description, context length, maximum output tokens, creation date, input/output modality, and tokenizer type. Create the author metadata file that exposes model count and support status to the registry.
Key considerations:
- Use exact API model IDs, not marketing names (e.g., "gpt-4-turbo-preview" not "GPT-4 Turbo")
- Model IDs should be found from actual API documentation, not pricing pages
- Modality specifies supported input types (text, image, audio) and output types
Step 5: Configure Endpoint Pricing
Define endpoint configurations that pair each model with one or more providers. Each endpoint specifies the provider-specific model ID, pricing tiers (with threshold-based rates for different context window sizes), rate limits, context length, maximum completion tokens, supported parameters, and deployment configurations for regional variants.
Key considerations:
- Pricing is always specified as cost per token (not per million tokens)
- Threshold-based pricing supports different rates above certain context lengths
- Cache read and write multipliers are specified relative to the input token cost
- Regional deployments (e.g., Bedrock us-east-1, us-west-2) get separate endpoint entries
Step 6: Update Central Registry
Import the new model definitions and endpoint configurations into the central registry files. Update both registry-types.ts (for TypeScript type derivation) and registry.ts (for runtime data). The registry builds O(1) lookup indexes at initialization for efficient model resolution during request routing.
Key considerations:
- Both registry-types.ts and registry.ts must be updated in sync
- The registry builds indexes for model-to-endpoints, provider-to-endpoints, and model-to-providers mappings
Step 7: Create Tests and Update Snapshots
Write test files covering the new provider's request routing, cost calculation, error handling, and edge cases. Run the snapshot tests to update the registry state snapshots that validate the complete model/provider configuration.
Key considerations:
- Tests should cover both PTB and BYOK request flows
- Error response mapping must be tested to ensure provider errors are properly normalized
- Registry snapshot tests must be regenerated after any model/endpoint changes