Principle:Helicone Helicone Backend Log Processing
| Knowledge Sources | |
|---|---|
| Domains | LLM Observability, Log Processing, Handler Chain |
| Last Updated | 2026-02-14 00:00 GMT |
Overview
Backend log processing is the technique of consuming queued log messages through a structured chain of handlers that authenticate, enrich, store, and distribute the log data across multiple systems.
Description
After the proxy layer publishes a log message to the queue, a backend service must consume and process that message. The processing is not a single operation but a multi-step pipeline where each step depends on the output of the previous one. The backend must authenticate the message (verifying that the Helicone API key or JWT is valid and resolving the organization), check rate limits, read request/response bodies from object storage if they were stored by reference, parse and normalize the bodies, handle prompt template extraction, compute token usage and cost, persist the final record to analytics storage, and distribute notifications to configured webhooks and third-party integrations (PostHog, Lytix, Segment, Stripe).
This pipeline is implemented using the Chain of Responsibility pattern, where each handler performs a discrete task, mutates a shared context object, and passes control to the next handler in the chain. If any handler encounters a fatal error, the chain can be short-circuited and the error reported.
The chain architecture provides several benefits: handlers can be added, removed, or reordered without modifying other handlers; each handler encapsulates a single concern; and the shared context object accumulates state progressively, making the final logging handler's job straightforward.
Usage
Use backend log processing when consuming telemetry messages from a queue and the processing involves multiple interdependent steps that must execute in a defined order. This pattern is appropriate when each step has its own failure modes and the system must handle partial failures gracefully (e.g., logging to the database should succeed even if a webhook delivery fails).
Theoretical Basis
The pattern implements a Chain of Responsibility combined with a Shared Context Accumulator.
The typical handler chain is:
- Authentication: Validate the authorization token, resolve the organization ID and access permissions.
- Rate Limiting: Record rate limit usage for the organization.
- S3 Reader: If request/response bodies were stored in object storage, fetch them.
- Request Body Handler: Parse and normalize the raw request body, extracting model information.
- Response Body Handler: Parse and normalize the raw response body, extracting token usage.
- Prompt Handler: If the request uses a managed prompt, extract template variables and version information.
- Online Evaluation: Run any configured online evaluators against the request/response pair.
- Stripe Integration: Apply billing-related property mutations before logging.
- Logging Handler: Persist the fully enriched record to ClickHouse and/or S3.
- PostHog / Lytix / Webhook / Segment / Stripe Log: Distribute notifications and analytics events to configured integrations.
Each handler receives a HandlerContext object that it reads from and writes to, then calls next.handle(context).