Neural Network Repair of Lossy Compression Artifacts in the September 2015 to March 2016 Duration of the MMS/FPI Data Set

Abstract

During the September 2015 to March 2016 duration (sometimes referred to as Phase 1A) of the Magnetospheric Multiscale Mission, the Dual Electron Spectrometers (DES) were configured to generously utilize lossy compression. While this maximized the number of velocity distribution functions downlinked, it came at the expense of lost information content for a fraction of the frames. Following this period of lossy compression, the DES was reconfigured in a way that allowed for 95% of the frames to arrive to the ground without loss. Using this high-quality set of frames from on-orbit observations, we compressed and decompressed the frames on the ground to create a side-by-side record of the compression effect. This record was used to drive an optimization method that (a) derived basis functions capable of approximating the lossless sample space and with nonnegative coefficients and (b) fitted a function which maps the lossy frames to basis weights that recreate the frame without compression artifacts. This method is introduced and evaluated in this paper. Data users should expect a higher level of confidence in the absolute scale of density/temperature measurements and notice less sinusoidal bias in the velocity X and Y components (GSE)