Robust Gaussian Mixture Modeling: A K Divergence Based Approach

Abstract

This paper addresses the problem of robust Gaussian mixture modeling in the presence of outliers. We commence by introducing a general expectation-maximization (EM)-like scheme, called K-BM, for iterative numerical computation of the minimum K-divergence estimator (MKDE). This estimator leverages Parzen's non-parametric Kernel density estimate to down-weight low density regions associated with outlying measurements. Akin to the conventional EM, the K-BM involves successive Maximizations of lower Bounds on the objective function of the MKDE. However, differently from EM, these bounds are not exclusively reliant on conditional expectations. The K-BM algorithm is applied to robust parameter estimation of a finite-order multivariate Gaussian mixture model (GMM). We proceed by introducing a new robust variant of the Bayesian information criterion (BIC) that penalizes the MKDE's objective function. The proposed criterion, called K-BIC, is conveniently applied for robust GMM order selection. In the paper, we also establish a data-driven procedure for selection of the kernel's bandwidth parameter. This procedure operates by minimizing an empirical asymptotic approximation of the mean-integrated-squared-error (MISE) between the underlying density and the estimated GMM density. Lastly, the K-BM, the K-BIC, and the MISE based selection of the kernel's bandwidth are combined into a unified framework for joint order selection and parameter estimation of a GMM. The advantages of the K-divergence based framework over other robust approaches are illustrated in simulation studies involving synthetic and real data.