Implementation:Helicone Helicone Registry Snapshot Tests

Knowledge Sources	Helicone
Domains	Testing, Model Registry, Jest
Last Updated	2026-02-14 00:00 GMT

Overview

Concrete Jest snapshot test suite that guards against unintended pricing, coverage, and configuration regressions in the model-provider registry, provided by the packages/__tests__/cost/registrySnapshots.test.ts module.

Description

The registrySnapshots.test.ts file dynamically discovers all endpoints.ts files under packages/cost/models/authors/ using glob, imports them at test time, and runs five distinct snapshot tests covering different facets of the registry state.

The test suite uses a beforeAll hook to asynchronously load all endpoint modules, extracting both active endpoints and archivedEndpoints exports from each file. This dynamic discovery approach ensures that new models are automatically included in the test suite without requiring manual test updates.

The five test cases are:

1. Pricing Snapshot -- Serializes the pricing arrays for every model grouped by provider and compares against a stored snapshot. Any change to per-token rates, cache multipliers, or modality pricing will cause this test to fail.

1. Model Coverage Snapshot -- Captures which providers serve each model (sorted alphabetically) and snapshots the result. Adding or removing a provider for a model requires an explicit snapshot update.

1. Endpoint Configurations Snapshot -- Captures the full structural configuration of each endpoint: provider, model ID, context length, max tokens, PTB enablement, cross-region flag, regions, and sorted parameter list.

1. Registry State Verification -- Flattens all configurations, builds the ModelIndexes using buildIndexes, and snapshots aggregate counts (total endpoints, total configs, total providers, PTB-enabled models), provider breakdowns, model-to-provider mappings, PTB-enabled model lists, and sample endpoint IDs. Also verifies that every PTB-enabled endpoint has a corresponding usage processor via getUsageProcessor.

1. Archived Endpoints Snapshot -- Captures versioned/archived configurations including provider, version, model ID, and pricing.

Usage

Run these tests as part of the standard test suite when modifying any model-provider configuration. When a snapshot test fails due to an intentional change, update the snapshots with npx jest --updateSnapshot (or jest -u) and commit the updated snapshot file alongside the configuration change.

Code Reference

Source Location

• Repository: Helicone
• File: packages/__tests__/cost/registrySnapshots.test.ts (lines 37-186)

Signature

// Dynamic endpoint loader
async function loadAllEndpoints(): Promise<{
  endpoints: Record<string, any>;
  archivedEndpoints: Record<string, any>;
}>;

describe("Registry Snapshots", () => {
  let allEndpoints: Record<string, any>;
  let allArchivedEndpoints: Record<string, any>;

  beforeAll(async () => {
    const loaded = await loadAllEndpoints();
    allEndpoints = loaded.endpoints;
    allArchivedEndpoints = loaded.archivedEndpoints;
  });

  it("pricing snapshot", () => { ... });
  it("model coverage snapshot", () => { ... });
  it("endpoint configurations snapshot", () => { ... });
  it("verify registry state", () => { ... });
  it("archived endpoints snapshot", () => { ... });
});

Import

// Internal imports used by the test file
import { buildIndexes } from "../../cost/models/build-indexes";
import { getUsageProcessor } from "@/cost/usage/getUsageProcessor";
import { sync as globSync } from "glob";

I/O Contract

Inputs

Name	Type	Required	Description
Endpoint files	**/endpoints.ts files	Yes	Dynamically discovered via glob from packages/cost/models/authors/; each must export an endpoints record and optionally an archivedEndpoints record
Stored snapshots	.snap file	Yes	Jest snapshot file committed to version control, serving as the baseline for comparison

Outputs

Name	Type	Description
Test result	Pass/Fail	Pass if all current snapshots match stored baselines; fail with a diff if any value has changed
Usage processor validation	Pass/Error	Throws an error if any PTB-enabled endpoint lacks a corresponding usage processor

Usage Examples

Basic Usage

// Run the snapshot tests from the packages directory
// npx jest __tests__/cost/registrySnapshots.test.ts

// When tests fail due to an intentional change, update snapshots:
// npx jest __tests__/cost/registrySnapshots.test.ts --updateSnapshot

// Example of what the pricing snapshot captures for a model:
// {
//   "anthropic/claude-opus-4": {
//     "anthropic": [
//       {
//         "threshold": 0,
//         "input": 0.000015,
//         "output": 0.000075,
//         "web_search": 0.01,
//         "cacheMultipliers": {
//           "cachedInput": 0.1,
//           "write5m": 1.25,
//           "write1h": 2.0
//         }
//       }
//     ],
//     "bedrock": [ ... ],
//     "vertex": [ ... ]
//   }
// }

// The verify registry state test also checks structural invariants:
// - Every PTB endpoint must have a usage processor
// - Total counts must match expectations
// - Provider breakdowns must be consistent

Related Pages

Implements Principle

• Principle:Helicone_Helicone_Registry_Snapshot_Testing