Workflow:SeldonIO Seldon core Model Explainability
| Knowledge Sources | |
|---|---|
| Domains | MLOps, Explainability, Data_Science, Kubernetes |
| Last Updated | 2026-02-13 14:00 GMT |
Overview
End-to-end process for adding model explainability to deployed ML models using Alibi Explain integration in Seldon Core 2.
Description
This workflow covers deploying explainer models that provide human-interpretable explanations for ML model predictions. Seldon Core 2 integrates with the Alibi Explain library to support explainability techniques including Anchor Tabular (for tabular data), Anchor Text (for NLP models), and Kernel SHAP (for feature importance). Explainers are deployed as separate Model resources that reference the base model they explain, enabling on-demand explanations without modifying the original prediction pipeline. The explainer queries the base model internally to generate explanations, making the process transparent to the end user.
Usage
Execute this workflow when you need to provide interpretable explanations for model predictions. Common triggers include regulatory compliance requirements (GDPR right to explanation), debugging model behavior on specific inputs, building trust with stakeholders, or investigating model failures. This workflow is appropriate when you have a deployed classification or regression model and need to understand which input features drive individual predictions.
Execution Steps
Step 1: Train Explainer Model
Create the explainer model artifact using the Alibi Explain library. For Anchor Tabular, train the explainer on representative data from the training set to build the perturbation space. For Anchor Text, configure the language model and perturbation strategy. For Kernel SHAP, define the background dataset for Shapley value computation. Save the trained explainer as a serialized artifact with its MLServer model-settings.json.
Key considerations:
- Anchor Tabular explainers need categorical feature mappings and the training data distribution
- Anchor Text explainers need a tokenizer and perturbation configuration
- Kernel SHAP explainers need a representative background dataset
- The explainer artifact must be compatible with MLServer's alibi-explain runtime
- Model-settings.json must specify the explainer type and the init_parameters
Step 2: Deploy Base Model
Ensure the base model that will be explained is deployed and serving predictions. The explainer will call this model internally during explanation generation, so it must be accessible and responsive.
Key considerations:
- The base model must accept the same input format that the explainer will use for perturbation queries
- The base model should be in the Available state before deploying the explainer
- Explainer generation involves many calls to the base model (hundreds to thousands), so the base model should have adequate throughput
Step 3: Deploy Explainer Model
Create and deploy the explainer Model resource with the explainer section configured. The explainer section specifies the explainability technique type and references the base model. The explainer is deployed as a regular Model resource but with the additional explainer metadata that connects it to its target model.
Key considerations:
- The explainer type field must match the trained explainer (anchor_tabular, anchor_text, etc.)
- The modelRef field must reference the deployed base model name
- Explainer models require the alibi-explain runtime capability
- Memory allocation for explainers should account for the internal perturbation data structures
Step 4: Verify Explainer Readiness
Wait for the explainer model to reach the Available state. Verify connectivity to the base model by checking that the explainer can successfully generate explanations for sample inputs.
Key considerations:
- Explainer loading may take longer than regular models due to initialization of the explanation framework
- Verify that the explainer can reach the base model's inference endpoint
- Test with a known input to validate the explanation output format
Step 5: Generate Explanations
Send inference requests to the explainer model endpoint to generate explanations for specific inputs. The explainer internally perturbs the input, queries the base model with perturbations, and returns an explanation describing which features most influence the prediction.
Key considerations:
- Explanation requests go to the explainer model endpoint, not the base model endpoint
- Explanation generation is computationally expensive (may take seconds to minutes per request)
- Anchor explanations return: the anchor features, precision, and coverage metrics
- SHAP explanations return: per-feature Shapley values indicating positive/negative contribution
- Explanations include examples of true/false positives in the anchor neighborhood
Step 6: Integrate with Pipeline
Optionally integrate the explainer into a broader pipeline that combines prediction with explanation. This can be done by adding the explainer as a pipeline step with custom input/output transforms, or by calling the explainer as a separate endpoint alongside the production pipeline.
Key considerations:
- Pipeline integration requires careful tensor mapping between the explainer and other steps
- Consider running explanations asynchronously to avoid increasing prediction latency
- For text models, custom output transforms may be needed to convert model outputs for the explainer
- Explainer results can be logged alongside predictions for audit trails