Principle:Langchain ai Langchain Streaming Routing
| Knowledge Sources | |
|---|---|
| Domains | Control_Flow, Streaming |
| Last Updated | 2026-02-11 00:00 GMT |
Overview
A decision mechanism that determines whether a model invocation should use the streaming path or the non-streaming path based on model capabilities and configuration.
Description
Not all chat models implement streaming, and some models may want to disable streaming in certain contexts (e.g., when tools are bound). Streaming routing inspects the model's capabilities at runtime to determine the optimal execution path:
- Whether _stream() or _astream() methods are overridden by the provider
- Whether disable_streaming is set (globally or for tool-calling contexts)
- Whether the caller explicitly requested streaming via kwargs
This ensures a graceful fallback: if streaming is not available, the model falls back to non-streaming invocation transparently.
Usage
This principle is applied automatically within the invoke() method. When called via invoke(), the framework checks if streaming should be used (for callback support). When called via stream(), streaming is always attempted if available.
Theoretical Basis
The routing decision follows a capability-based dispatch pattern:
# Abstract algorithm (not real code)
def should_stream(model, async_api, **kwargs):
if model.disable_streaming is True:
return False
if model.disable_streaming == "tool_calling" and tools_are_bound(kwargs):
return False
if async_api:
return has_override(model, "_astream") or has_override(model, "_stream")
else:
return has_override(model, "_stream")