Semi-supervised Expectation Maximization with Contrastive Outlier Removal.
Loading...
Links to Files
Permanent Link
Author/Creator
Author/Creator ORCID
Date
2022-01-01
Type of Work
Department
Computer Science and Electrical Engineering
Program
Computer Science
Citation of Original Publication
Rights
This item may be protected under Title 17 of the U.S. Copyright Law. It is made available by UMBC for non-commercial research and education. For permission to publish or reproduce, please see http://aok.lib.umbc.edu/specoll/repro.php or contact Special Collections at speccoll(at)umbc.edu
Distribution Rights granted to UMBC by the author.
Access limited to the UMBC community. Item may possibly be obtained via Interlibrary Loan thorugh a local library, pending author/copyright holder's permission.
Distribution Rights granted to UMBC by the author.
Access limited to the UMBC community. Item may possibly be obtained via Interlibrary Loan thorugh a local library, pending author/copyright holder's permission.
Abstract
Semi-supervised learning has proven to be one of the most widely used techniques to overcome the concern of limited labels. One of the concerns while using neural networks for semi-supervised learning in presence of an extremely small labeled dataset is the occurrence of confidently predicted incorrect labels. This phenomenon of confidently predicting incorrect labels for unsupervised data is called confounding bias. Even though pseudo-labeling and consistency regularization are among the state-of-the-art techniques for semi-supervised learning, these techniques are susceptible to the problem of confounding bias while using neural networks. We propose a methodology that could help neural networks overcome this problem by leveraging information from unlabeled images using cluster generating techniques and smoothness generating techniques in a tightly-coupled way to overcome the fundamental problem of outliers. These techniques could help the model to learn certain attributes from the image which could not be learned from the original resolution of the unlabeled images. We argue both theoretically and empirically that contrastive outlier suppression is a necessary yet overlooked criteria in the application of EM-derived latent bootstrapping, because discrimination models such as neural networks have the potential to make erronous predictions with high confidence if these datasets are far from the decision boundary, whereas generative methods for which Expectation Maximization (EM) was originally designed have no such issue. Contrastive outlier suppression is derived under the assumption that the latent feature vector predictions should follow a multivariate gaussian mixture distribution. Our results show that contrastive latent bootstrapping greatly improves semi-supervised classification accuracy over a baseline, and furthermore when combined with a state-of-the-art consistency regularization method, our results achieve the highest reported semi-supervised accuracy for the CIFAR-10 classification using only 250 labeled sample images.