Principle:Guardrails ai Guardrails Stream Chunk Processing
| Knowledge Sources | |
|---|---|
| Domains | Streaming, Validation |
| Last Updated | 2026-02-14 00:00 GMT |
Overview
A chunk processing principle that manages the accumulation, boundary detection, and incremental validation of streaming LLM output.
Description
Stream Chunk Processing is the internal mechanism that bridges raw LLM stream chunks and validated output segments. The StreamRunner receives individual tokens or small text fragments from the LLM stream, accumulates them into a buffer, applies the validator's chunking strategy to detect segment boundaries (typically sentences), validates completed segments, and yields results while carrying incomplete text forward.
For string output, chunks are split at sentence boundaries using tokenization utilities. For JSON output, progressive parsing validates partial JSON structures as they form.
Usage
This principle is applied internally by the StreamRunner. Users interact with it indirectly through the streaming Guard call. Custom validators can influence chunk processing by overriding the _chunking_function method.
Theoretical Basis
The chunk processing pipeline:
- Receive: Get raw chunk from LLM stream (token or ChatCompletionChunk)
- Accumulate: Append to running buffer string
- Detect: Apply _chunking_function(buffer) to check for complete segments
- Validate: Run validators on the complete segment
- Yield: Produce ValidationOutcome with validated segment
- Carry: Store incomplete remainder for next cycle
For string output, the default boundary detection uses sentence tokenization via split_sentence_word_tokenizers_jl_separator.