Backdoor Attacks on Vision Transformers

Thumbnail Image

Files

2206.08477.pdf (8.02 MB)

Links to Files

https://arxiv.org/abs/2206.08477

Permanent Link

https://doi.org/10.48550/arXiv.2206.08477
 http://hdl.handle.net/11603/25153

Collections

UMBC Computer Science and Electrical Engineering Department
UMBC Student Collection

Author/Creator

Author/Creator ORCID

https://orcid.org/0000-0002-5394-7172

Date

2022-06-16

Type of Work

journal articles
preprints
Text

Department

Program

Citation of Original Publication

Rights

This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
Attribution 4.0 International (CC BY 4.0)

Subjects

Abstract

Vision Transformers (ViT) have recently demonstrated exemplary performance on a variety of vision tasks and are being used as an alternative to CNNs. Their design is based on a self-attention mechanism that processes images as a sequence of patches, which is quite different compared to CNNs. Hence it is interesting to study if ViTs are vulnerable to backdoor attacks. Backdoor attacks happen when an attacker poisons a small part of the training data for malicious purposes. The model performance is good on clean test images, but the attacker can manipulate the decision of the model by showing the trigger at test time. To the best of our knowledge, we are the first to show that ViTs are vulnerable to backdoor attacks. We also find an intriguing difference between ViTs and CNNs - interpretation algorithms effectively highlight the trigger on test images for ViTs but not for CNNs. Based on this observation, we propose a test-time image blocking defense for ViTs which reduces the attack success rate by a large margin.