High dimensional, robust, unsupervised record linkage
Loading...
Links to Files
Collections
Author/Creator
Author/Creator ORCID
Date
2020
Type of Work
Department
Program
Citation of Original Publication
Bera, Sabyasachi, and Snigdhansu Chatterjee. “High Dimensional, Robust, Unsupervised Record Linkage.” Statistics in Transition New Series 21, no. 4 (2020): 123–43. https://doi.org/10.21307/stattrans-2020-034.
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International CC BY-NC-ND 4.0 Deed
Subjects
Abstract
We develop a technique for record linkage on high dimensional data, where the two datasets may not have any common variable, and there may be no training set available. Our methodology is based on sparse, high dimensional principal components. Since large and high dimensional datasets are often prone to outliers and aberrant observations, we propose a technique for estimating robust, high dimensional principal components. We present theoretical results validating the robust, high dimensional principal component estimation steps, and justifying their use for record linkage. Some numeric results and remarks are also presented.