Locality Preserving Loss: Neighbors that Live together, Align together
| dc.contributor.author | Ganesan, Ashwinkumar | |
| dc.contributor.author | Ferraro, Francis | |
| dc.contributor.author | Oates, Tim | |
| dc.date.accessioned | 2021-08-09T13:59:36Z | |
| dc.date.available | 2021-08-09T13:59:36Z | |
| dc.date.issued | 2021-04-20 | |
| dc.description | Proceedings of the Second Workshop on Domain Adaptation for NLP, pages 122–139 April 20, 2021 | en_US |
| dc.description.abstract | We present a locality preserving loss (LPL) that improves the alignment between vector space embeddings while separating uncorrelated representations. Given two pretrained embedding manifolds, LPL optimizes a model to project an embedding and maintain its local neighborhood while aligning one manifold to another. This reduces the overall size of the dataset required to align the two in tasks such as crosslingual word alignment. We show that the LPL-based alignment between input vector spaces acts as a regularizer, leading to better and consistent accuracy than the baseline, especially when the size of the training set is small. We demonstrate the effectiveness of LPL-optimized alignment on semantic text similarity (STS), natural language inference (SNLI), multi-genre language inference (MNLI) and cross-lingual word alignment (CLA) showing consistent improvements, finding up to 16% improvement over our baseline in lower resource settings. | en_US |
| dc.description.sponsorship | We would like to thank our anonymous reviewers from the Adapt-NLP Workshop and previous NLP conferences for their constructive reviews. We thank Prof. Konstantinos Kalpakis for his insights about manifold alignment and locality preservation methods. The hardware used in our computational studies is part of the UMBC HPCF facility and UMBC’s CARTA lab. This material is also based on research that is in part supported by the Air Force Research Laboratory (AFRL), DARPA, for the KAIROS program under agreement number FA8750-19-2-1003. The U.S.Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either express or implied, of the Air Force Research Laboratory (AFRL), DARPA, or the U.S. Government. | en_US |
| dc.description.uri | https://aclanthology.org/2021.adaptnlp-1.13/ | en_US |
| dc.format.extent | 18 pages | en_US |
| dc.genre | conference papers and proceedings | en_US |
| dc.identifier | doi:10.13016/m2x4cb-ju7a | |
| dc.identifier.citation | Ganesan, Ashwinkumar; Ferraro, Francis; Oates, Tim; Locality Preserving Loss: Neighbors that Live together, Align together; Proceedings of the Second Workshop on Domain Adaptation for NLP, 20 April, 2021; https://aclanthology.org/2021.adaptnlp-1.13/ | en_US |
| dc.identifier.uri | http://hdl.handle.net/11603/22339 | |
| dc.language.iso | en_US | en_US |
| dc.publisher | Association for Computational Linguistics | en_US |
| dc.relation.isAvailableAt | The University of Maryland, Baltimore County (UMBC) | |
| dc.relation.ispartof | UMBC Computer Science and Electrical Engineering Department Collection | |
| dc.relation.ispartof | UMBC Faculty Collection | |
| dc.rights | This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author. | en_US |
| dc.subject | UMBC Ebiquity Research Group | en_US |
| dc.subject | UMBC High Performance Computing Facility (HPCF) | |
| dc.title | Locality Preserving Loss: Neighbors that Live together, Align together | en_US |
| dc.type | Text | en_US |