Machine Unlearning and Model Editing

Abstract

The process of making a model ”forget” specific data, machine unlearning ensures important data that has had its access revoked can be forgotten. This is especially important if the data the model is being trained on raises security and ethics concerns, like the GDPR mandate that gives individuals the ”right to be forgotten”. In order to comply with such ethical and legal responsibilities, there has to be some method in place to remove data that is considered to violate these conventions. Additionally, such techniques are also vital in making sure erroneous data that skews the model’s accuracy and fairness can be removed to make the model better at the task at hand without retraining it completely. This is challenging because once the model learns data, it becomes difficult to ”un-entangle” it from the parameters, and removing the entanglement is a challenge. This paper aims to study the field of machine unlearning in depth, starting from the origins and theory behind this idea and then summarizing all the state-of-the-art methods in this field. Finally, we discuss some of the problems that lay ahead in research in this field, and how researchers can hope to resolve them.