Principle:Microsoft Semantic kernel Observability Filters
| Knowledge Sources | |
|---|---|
| Domains | AI_Orchestration, Observability, Middleware |
| Last Updated | 2026-02-11 19:00 GMT |
Overview
Middleware-style interceptors enable observing, logging, modifying, and short-circuiting function invocations and prompt rendering within an AI orchestration pipeline.
Description
Observability Filters are the principle of providing structured extension points at critical stages of the AI orchestration pipeline, allowing developers to intercept, observe, and modify behavior without changing the core framework or plugin code. This principle applies the middleware pattern (familiar from ASP.NET Core's request pipeline) to AI function invocation and prompt rendering.
Semantic Kernel defines three filter interfaces, each targeting a different stage of the pipeline:
- IFunctionInvocationFilter: Intercepts every function invocation (both native plugin functions and prompt functions). The filter receives a FunctionInvocationContext containing the kernel, the function being called, its arguments, and its result. By calling or not calling the next delegate, the filter can proceed with invocation, skip it, or modify the arguments or result.
- IPromptRenderFilter: Intercepts the prompt rendering stage that occurs before the rendered prompt is sent to the AI model. The filter receives a PromptRenderContext containing the kernel, function, arguments, execution settings, and the rendered prompt text. The filter can modify the rendered prompt (e.g., for content filtering, PII redaction, or logging) or short-circuit by setting a Result directly.
- IAutoFunctionInvocationFilter: Specifically intercepts automatic function invocations triggered by AI model function call requests during the auto-invoke loop. This filter receives an AutoFunctionInvocationContext with additional metadata about the AI-initiated function call.
All three filter types follow the same delegation pattern: the filter method receives a context and a next delegate. Calling next passes control to the next filter in the pipeline (or to the actual operation if this is the last filter). Not calling next short-circuits the pipeline, skipping subsequent filters and the operation itself.
Multiple filters can be registered and are invoked in registration order, forming a pipeline. This composable design allows separation of concerns: one filter for logging, another for telemetry, another for content safety, each operating independently.
Usage
Use observability filters for cross-cutting concerns that should apply to all (or many) function invocations or prompt renderings. Common use cases include:
- Logging and telemetry: Recording function names, arguments, results, and execution times.
- Content safety: Inspecting or modifying rendered prompts before they reach the AI model.
- PII redaction: Stripping sensitive data from prompts or function results.
- Cost tracking: Counting AI model invocations and function calls for billing.
- Short-circuiting: Returning cached results or blocking certain function calls based on policy.
- Error handling: Wrapping function invocations in try-catch logic and providing fallback results.
Theoretical Basis
Observability Filters implement the Chain of Responsibility pattern (also known as the middleware pipeline pattern). Each filter is a handler in the chain that can process the request, modify it, pass it along, or terminate the chain.
Formal model:
Let F1, F2, ..., Fn be a sequence of registered filters and Op be the actual operation (function invocation or prompt rendering). The pipeline executes as:
Pipeline(context):
F1(context, () =>
F2(context, () =>
...
Fn(context, () =>
Op(context)
)
...
)
)
Each filter Fi receives the context and a next delegate. The filter can:
// Proceed normally
await next(context);
// Modify arguments before proceeding
context.Arguments["key"] = newValue;
await next(context);
// Modify result after proceeding
await next(context);
context.Result = modifiedResult;
// Short-circuit (skip remaining filters and the operation)
context.Result = cachedResult;
// (do not call next)
Key invariants:
- Ordering: Filters execute in registration order. The first registered filter is the outermost wrapper.
- Opt-in delegation: A filter must explicitly call next to proceed. Not calling next is a valid and intentional short-circuit.
- Context mutability: The context object is mutable. Modifications made by one filter (to arguments, result, or rendered prompt) are visible to subsequent filters and the operation itself.
- Async throughout: All filter methods are async (Task-returning), supporting asynchronous cross-cutting concerns like remote logging or async cache lookups.
- Scope: IFunctionInvocationFilter applies to all function invocations. IPromptRenderFilter applies only to prompt rendering. IAutoFunctionInvocationFilter applies only to AI-initiated automatic function calls.