Heuristic:Helicone Helicone Rate Limiting Fail Open
| Knowledge Sources | |
|---|---|
| Domains | Proxy_Resilience, Rate_Limiting |
| Last Updated | 2026-02-14 06:00 GMT |
Overview
Rate limiting in the Helicone proxy uses a fail-open strategy: errors in rate limit checks do not block requests, preserving proxy availability over strict enforcement.
Description
The Helicone proxy sits between customers and their LLM providers. If the rate limiting subsystem (implemented via Cloudflare Durable Objects) encounters an error (network timeout, DO crash, KV read failure), the proxy allows the request to pass through rather than returning a rate-limit error. This design choice prioritizes availability over strict rate enforcement: it is better to occasionally allow a request above the rate limit than to block legitimate requests due to infrastructure failures.
Usage
Apply this heuristic when implementing or modifying rate limiting logic in the proxy worker. Any new rate limiting mechanism should default to allowing requests on error. Only explicit `!allowed` signals from a healthy rate limiter should trigger HTTP 429 responses.
The Insight (Rule of Thumb)
- Action: Configure rate limiters with `failureMode: "fail-open"`. Wrap rate limit checks in try/catch blocks that silently continue on error.
- Value: Errors in rate limiting infrastructure are logged but do not block requests.
- Trade-off: Under rate limiter failures, rate limits are not enforced. This can lead to temporary overuse but prevents outages for all customers.
Reasoning
Code evidence from `worker/src/lib/HeliconeProxyRequest/ProxyForwarder.ts`:
if (useBucketRateLimiter && effectivePolicyHeader && !rateLimited) {
try {
const bucketResult = await checkBucketRateLimit({
policyHeader: effectivePolicyHeader,
organizationId: orgData.organizationId,
userId: proxyRequest.userId,
heliconeProperties: proxyRequest.heliconeProperties,
rateLimiterDO: env.BUCKET_RATE_LIMITER,
config: {
failureMode: "fail-open", // Preserve availability on errors
},
});
if (!bucketResult.allowed) {
rateLimited = true;
}
} catch {
// Rate limit check failed - fail open for availability
}
}
The `failureMode: "fail-open"` configuration and the empty `catch` block together implement the fail-open pattern. This is a deliberate design decision documented via the configuration parameter name itself.
The same pattern applies to the legacy rate limiter path, where authentication or organization lookup failures cause the rate limiting check to be skipped entirely rather than blocking the request.