When Explanations Lie: Why Many Modified BP Attributions Fail

December 20, 2019 · Entered Twilight · 🏛 International Conference on Machine Learning

"Last commit was 6.0 years ago (≥5 year threshold)"

Evidence collected by the PWNC Scanner

Repo contents: .gitignore, README.md, conda_list_output.txt, convergence_simulation.ipynb, cosine_similarity_convergence.ipynb, create_code_submission.sh, deeplift_resnet.py, development, imagenet_dir.json, imagenet_dir.json.template, imagenet_labels.json, images, monkey_patch_lrp_resnet.py, pattern_attr.ipynb, sanity_checks.ipynb, train_cifar10.ipynb, two_classes.ipynb, when_explanations_lie.py

Authors Leon Sixt, Maximilian Granz, Tim Landgraf arXiv ID 1912.09818 Category cs.LG: Machine Learning Cross-listed stat.ML Citations 145 Venue International Conference on Machine Learning Repository https://github.com/berleon/when-explanations-lie ⭐ 6 Last Checked 2 months ago

Abstract

Attribution methods aim to explain a neural network's prediction by highlighting the most relevant image areas. A popular approach is to backpropagate (BP) a custom relevance score using modified rules, rather than the gradient. We analyze an extensive set of modified BP methods: Deep Taylor Decomposition, Layer-wise Relevance Propagation (LRP), Excitation BP, PatternAttribution, DeepLIFT, Deconv, RectGrad, and Guided BP. We find empirically that the explanations of all mentioned methods, except for DeepLIFT, are independent of the parameters of later layers. We provide theoretical insights for this surprising behavior and also analyze why DeepLIFT does not suffer from this limitation. Empirically, we measure how information of later layers is ignored by using our new metric, cosine similarity convergence (CSC). The paper provides a framework to assess the faithfulness of new and existing modified BP methods theoretically and empirically. For code see: https://github.com/berleon/when-explanations-lie