Principle:SeldonIO Seldon core Explanation Generation
| Field | Value |
|---|---|
| Overview | Generating human-interpretable explanations for individual predictions by querying explainer endpoints. |
| Domains | Explainability, Inference |
| Workflow | Model_Explainability |
| Related Implementation | SeldonIO_Seldon_core_Seldon_Model_Infer_Explainer |
| Last Updated | 2026-02-13 00:00 GMT |
Description
Explanation generation sends an inference request to the explainer model endpoint (not the base model). The explainer internally queries the base model multiple times with perturbed inputs to build the explanation. The response includes anchor features, precision, coverage, and detailed perturbation results. For tabular data, the explanation identifies which feature conditions guarantee the prediction.
The explanation flow operates as follows:
- The client sends a V2 inference request to the explainer endpoint with the input instance to explain.
- The Alibi-Explain runtime on MLServer receives the request and invokes the loaded explainer algorithm.
- The explainer generates hundreds to thousands of perturbed versions of the input.
- Each perturbed input is sent to the base model (via
modelRef) for prediction. - The explainer analyzes the prediction patterns to construct the explanation.
- The explanation is returned to the client as a V2 inference response with structured metadata.
For AnchorTabular, the explanation identifies feature conditions (e.g., "Age > 37 AND Capital Gain <= 0") that guarantee the prediction with high probability. For AnchorText, it identifies words whose presence anchors the prediction. For KernelShap, it returns per-feature importance scores.
Theoretical Basis
Anchor explanation generation is a beam search over feature predicates. The algorithm iteratively expands candidate anchors by adding feature conditions, evaluating each candidate's precision via Monte Carlo sampling. The search terminates when precision exceeds the threshold τ.
The beam search operates as follows:
- Start with an empty anchor (no conditions).
- Generate candidate expansions by adding one feature condition.
- For each candidate, sample n perturbations that satisfy the anchor conditions.
- Send perturbations to the base model and compute precision: the fraction where the prediction matches the original.
- Retain the top-B candidates (beam width) and repeat.
- Stop when a candidate exceeds precision threshold τ (default 0.95).
The returned anchor is the minimal set of conditions sufficient for the prediction. The coverage metric indicates what fraction of the dataset the anchor applies to, providing a measure of the explanation's generality.
Usage
When requesting interpretable explanations for individual model predictions in production. Explanation requests are typically made for specific instances of interest (e.g., flagged predictions, edge cases, or auditing samples) rather than for every prediction.
Knowledge Sources
Related Pages
- SeldonIO_Seldon_core_Seldon_Model_Infer_Explainer - implements this principle - Concrete CLI tool for generating explanations via inference requests.
- SeldonIO_Seldon_core_Explainer_Readiness_Verification - prerequisite - The explainer must be confirmed ready before sending explanation requests.
- SeldonIO_Seldon_core_Explainer_Model_Training - related principle - The explainer's behavior is determined by its training configuration.
Implementation:SeldonIO_Seldon_core_Seldon_Model_Infer_Explainer