Workflow:Interpretml Interpret Blackbox Model Explanation
| Knowledge Sources | |
|---|---|
| Domains | Machine_Learning, Interpretability, Model_Agnostic_Explanations |
| Last Updated | 2026-02-07 12:00 GMT |
Overview
End-to-end process for explaining any blackbox machine learning model's predictions using model-agnostic explanation techniques (SHAP, LIME, Partial Dependence, Sensitivity Analysis) through the InterpretML unified interface.
Description
This workflow enables users to explain predictions from any machine learning model, regardless of its internal structure, using the InterpretML blackbox explainer suite. The framework provides a unified API that wraps established explanation methods: SHAP KernelExplainer for Shapley-value-based feature attributions, LIME for local surrogate model explanations, Partial Dependence Plots for marginal feature effects, and Morris Sensitivity Analysis for global feature importance. All explainers follow the same ExplainerMixin contract, producing Explanation objects that integrate seamlessly with the InterpretML visualization system.
Key outputs:
- Local explanations: per-instance feature attribution scores
- Global explanations: feature importance rankings and marginal effect plots
- Visualization-ready explanation objects compatible with show()
Scope:
- Covers model-agnostic explanation for any predict/predict_proba callable
- Includes SHAP, LIME, Partial Dependence, and Morris Sensitivity methods
- Supports binary classification and regression (SHAP/LIME limited to binary classification)
Strategy:
- Wrap existing explanation libraries (shap, lime, SALib) behind a unified interface
- Standardize output format so all explainers produce compatible Explanation objects
- Leverage the same visualization pipeline as glassbox models
Usage
Execute this workflow when you have a trained blackbox model (random forest, gradient boosting, neural network, etc.) and need to understand its predictions. Use SHAP for theoretically grounded feature attributions based on game theory, LIME for quick local approximations using interpretable surrogate models, Partial Dependence for understanding marginal feature effects, or Morris Sensitivity for efficient global sensitivity screening. This is particularly useful for model debugging, fairness auditing, or satisfying regulatory requirements when the model itself is not inherently interpretable.
Execution Steps
Step 1: Prepare the Model and Reference Data
Initialize the blackbox explainer with the trained model and a reference dataset. The model must expose a prediction function (predict or predict_proba). The reference dataset serves as the background distribution for computing expected values (SHAP) or for generating perturbations (LIME, Sensitivity). Data is cleaned and converted to float64 format as required by the underlying explanation libraries.
Key considerations:
- The model parameter accepts any callable with predict/predict_proba interface
- Reference data should be representative of the training distribution
- Feature names and types can be explicitly specified or auto-detected
- For SHAP: the reference data defines the baseline expectation
- For LIME: the reference data trains the internal feature statistics
- SHAP and LIME currently support binary classification and regression only
Step 2: Select Explanation Method
Choose the appropriate explanation technique based on the analysis goal. Each method has different strengths: SHAP provides theoretically consistent attributions, LIME offers fast local approximations, Partial Dependence shows average marginal effects, and Morris Sensitivity efficiently screens feature importance.
Available methods:
- ShapKernel: Model-agnostic SHAP using kernel approximation. Produces Shapley values as feature attributions. Best for precise per-instance explanations but computationally expensive.
- LimeTabular: Local Interpretable Model-agnostic Explanations. Fits a weighted linear model around each instance. Fast but approximations may vary between runs.
- PartialDependence: Shows the marginal effect of features on predictions by averaging over the reference data. Best for understanding global feature effects.
- MorrisSensitivity: Elementary effects screening method. Efficiently identifies which features have the largest influence on model output using a one-at-a-time design.
Step 3: Generate Explanations
Call the appropriate explain method on the initialized explainer. For local explanations (SHAP, LIME), pass the specific instances to explain. For global explanations (Partial Dependence, Sensitivity), the method analyzes the model across the feature space using the reference data.
What happens:
- Local (SHAP): For each instance, computes Shapley values by evaluating the model on masked feature combinations. Returns per-feature attribution scores that sum to the difference between the prediction and the expected value.
- Local (LIME): For each instance, generates perturbed samples around the instance, gets model predictions, and fits a weighted sparse linear model. Returns feature weights from the local surrogate.
- Global (PDP): For each feature, varies its value across the observed range while marginalizing over other features. Returns the average prediction at each feature value.
- Global (Sensitivity): Applies Morris method with random trajectories through the feature space. Returns mean elementary effects and their standard deviations per feature.
Step 4: Visualize Explanations
Pass the generated explanation objects to the InterpretML show() function for interactive visualization. The visualization system produces the same types of charts for blackbox explanations as for glassbox models, enabling direct comparison.
Key considerations:
- Local explanations render as horizontal bar charts of feature contributions
- Global explanations render as importance bar charts or line plots
- Multiple explanations can be compared in a single dashboard using show([exp1, exp2])
- The same environment detection and provider selection applies as for EBM explanations