Consistent Explanations by Contrastive Learning
| dc.contributor.author | Pillai, Vipin | |
| dc.contributor.author | Koohpayegani, Soroush Abbasi | |
| dc.contributor.author | Ouligian, Ashley | |
| dc.contributor.author | Fong, Dennis | |
| dc.contributor.author | Pirsiavash, Hamed | |
| dc.date.accessioned | 2021-11-04T15:53:06Z | |
| dc.date.available | 2021-11-04T15:53:06Z | |
| dc.date.issued | 2021-10-01 | |
| dc.description | 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) | |
| dc.description.abstract | Understanding and explaining the decisions of neural networks are critical to building trust, rather than relying on them as black box algorithms. Post-hoc evaluation techniques, such as Grad-CAM, enable humans to inspect the spatial regions responsible for a particular network decision. However, it is shown that such explanations are not always consistent with human priors, such as consistency across image transformations. Given an interpretation algorithm, e.g., Grad-CAM, we introduce a novel training method to train the model to produce more consistent explanations. Since obtaining the ground truth for a desired model interpretation is not a well-defined task, we adopt ideas from contrastive self-supervised learning and apply them to the interpretations of the model rather than its embeddings. Explicitly training the network to produce more reasonable interpretations and subsequently evaluating those interpretations will enhance our ability to trust the network. We show that our method, Contrastive Grad-CAM Consistency (CGC), results in Grad-CAM interpretation heatmaps that are consistent with human annotations while still achieving comparable classification accuracy. Moreover, since our method can be seen as a form of regularizer, on limited-data fine-grained classification settings, our method outperforms the baseline classification accuracy on Caltech-Birds, Stanford Cars, VGG Flowers, and FGVC-Aircraft datasets. In addition, because our method does not rely on annotations, it allows for the incorporation of unlabeled data into training, which enables better generalization of the model. Our code is publicly available. | en_US |
| dc.description.sponsorship | : This material is based upon work partially supported by the United States Air Force under Contract No. FA8750-19-C-0098, U.S. Department of Commerce, National Institute of Standards and Technology under award number 60NANB18D279, NSF grant numbers 1845216 and 1920079, and funding from Northrop Grumman and SAP SE. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the United States Air Force, DARPA, or other funding agencies | en_US |
| dc.description.uri | https://www.computer.org/csdl/proceedings-article/cvpr/2022/694600k0203/1H1kRR6oMx2 | en_US |
| dc.format.extent | 11 pages | en_US |
| dc.genre | conference papers and proceedings | en_US |
| dc.genre | preprints | en_US |
| dc.identifier | doi:10.13016/m2qwcu-fj3k | |
| dc.identifier.citation | V. Pillai, S. Koohpayegani, A. Ouligian, D. Fong and H. Pirsiavash, "Consistent Explanations by Contrastive Learning," in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022 pp. 10203-10212. doi: 10.1109/CVPR52688.2022.00997 | |
| dc.identifier.uri | http://hdl.handle.net/11603/23218 | |
| dc.identifier.uri | https://doi.ieeecomputersociety.org/10.1109/CVPR52688.2022.00997 | |
| dc.language.iso | en_US | en_US |
| dc.publisher | IEEE | |
| dc.relation.isAvailableAt | The University of Maryland, Baltimore County (UMBC) | |
| dc.relation.ispartof | UMBC Computer Science and Electrical Engineering Department Collection | |
| dc.relation.ispartof | UMBC Student Collection | |
| dc.relation.ispartof | UMBC Faculty Collection | |
| dc.rights | © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | |
| dc.title | Consistent Explanations by Contrastive Learning | en_US |
| dc.type | Text | en_US |