Model Explainability (SHAP/LIME) Cheat Sheet
Interpret black-box model predictions using SHAP Shapley values and LIME local surrogate models, with plots and feature attribution.
SHAP TreeExplainer
Compute Shapley values efficiently for tree-based models like XGBoost or LightGBM.
import shapimport xgboost as xgbmodel = xgb.XGBClassifier().fit(X_train, y_train)explainer = shap.TreeExplainer(model)shap_values = explainer.shap_values(X_test)# force plot for a single predictionshap.force_plot(explainer.expected_value, shap_values[0], X_test.iloc[0])
Global Feature Importance Plots
Summarize feature impact across the whole test set with beeswarm and bar plots.
shap.summary_plot(shap_values, X_test) # beeswarm: direction + magnitude per featureshap.summary_plot(shap_values, X_test, plot_type="bar") # mean |SHAP value| ranking# dependence plot: how one feature's value affects its SHAP contributionshap.dependence_plot("age", shap_values, X_test)
Model-Agnostic Explanations with KernelExplainer
Explain any black-box model (e.g. an sklearn pipeline) using sampled background data.
background = shap.sample(X_train, 100)explainer = shap.KernelExplainer(model.predict_proba, background)shap_values = explainer.shap_values(X_test.iloc[:20], nsamples=200)shap.summary_plot(shap_values[1], X_test.iloc[:20]) # class 1
Local Explanation with LIME
Explain a single prediction by fitting an interpretable local surrogate model.
from lime.lime_tabular import LimeTabularExplainerexplainer = LimeTabularExplainer( X_train.values, feature_names=X_train.columns.tolist(), class_names=["no_churn", "churn"], mode="classification",)exp = explainer.explain_instance( X_test.iloc[0].values, model.predict_proba, num_features=8)exp.show_in_notebook()print(exp.as_list())
SHAP vs LIME
When to reach for each explainability approach.
- SHAP- theoretically grounded (Shapley values), consistent global + local attributions
- TreeExplainer- exact, fast SHAP values for tree ensembles specifically
- KernelExplainer- model-agnostic but slow; approximates SHAP via weighted sampling
- LIME- fast local surrogate, easier to reason about but less stable across runs
- Additivity property- SHAP values sum to (prediction - baseline), which LIME does not guarantee
Unified shap.Explainer API
Let SHAP auto-select the fastest exact/approximate algorithm for the model type instead of picking TreeExplainer/KernelExplainer manually.
explainer = shap.Explainer(model, X_train) # picks Tree/Linear/Permutation as appropriateshap_values = explainer(X_test)shap.plots.waterfall(shap_values[0]) # single-prediction contribution breakdownshap.plots.beeswarm(shap_values) # dataset-wide summary, same object as summary_plot
Pairwise Feature Interactions
Decompose SHAP values further into main effects and pairwise interaction effects for tree models.
explainer = shap.TreeExplainer(model)interaction_values = explainer.shap_interaction_values(X_test)# shape: (n_samples, n_features, n_features)shap.summary_plot(interaction_values, X_test)shap.dependence_plot(("age", "income"), interaction_values, X_test)
Explain a Text Classifier's Predictions
Use PartitionExplainer to attribute a transformer model's output to individual input tokens.
import shapfrom transformers import pipelineclassifier = pipeline( "text-classification", model="distilbert-base-uncased-finetuned-sst-2-english", top_k=None,)explainer = shap.Explainer(classifier)shap_values = explainer(["This movie was surprisingly good."])shap.plots.text(shap_values[0]) # highlights each token's contribution
LIME for Text Models
Apply LIME's text-specific explainer when the model consumes raw strings rather than tabular features.
from lime.lime_text import LimeTextExplainerexplainer = LimeTextExplainer(class_names=["negative", "positive"])exp = explainer.explain_instance( "This movie was surprisingly good.", classifier_fn, # callable: list[str] -> ndarray of class probabilities num_features=10,)exp.show_in_notebook(text=True)print(exp.as_list())
Common Explainability Pitfalls
Ways SHAP/LIME output gets misread in production reviews.
- Correlated features- credit splits arbitrarily between correlated columns; a high SHAP value on one doesn't mean the other is unimportant
- Correlation vs causation- SHAP explains what the model learned, not the real-world causal driver of the outcome
- Background data leakage- using training data as the KernelExplainer/DeepExplainer background can leak label information into the baseline
- Additivity misread- individual SHAP values only sum to (prediction - expected_value); comparing raw magnitudes across differently-scaled models is meaningless
- LIME instability- local surrogate fits vary run-to-run due to random perturbation sampling; average over several calls before trusting a single explanation
- Class imbalance skew- the SHAP baseline (expected_value) reflects the training class balance, so explanations can look misleading on a rebalanced serving population
Use TreeExplainer whenever the model is tree-based — it computes exact SHAP values in polynomial time, whereas KernelExplainer only approximates them and can give unstable results run-to-run on the same input.
