Towards Developing Scalable Activity Recognition Approach in IoT

Abstract

Technological advancement has increased the impact of wearable smart devices (such as smartphones, smartwatches) in our everyday lives. Wearable smart devices can capture a large volume of data that is often unlabeled, and labeling such large quantities of data is tedious and error-prone. In addition, the mismatch of the training and test data distributions makes it even more complicated for the machine learning models performance in finding useful information from this large volume of data. Transfer learning-based approaches such as Domain Adaptation are well known to tackle the implications of the heterogeneity of the data distribution between the training and test datasets. As a transfer learning-based approach, domain adaptation techniques leverage an existing labeled data source; however, there are several underpinning limitations in the majority of the existing methodologies: 1) they inherently assume that the unlabeled test data source (target) has high relevancy to the aiding labeled data source; 2) they are often unaccommodating to multiple aiding data sources (source) where the target data source could have relevancy with multiple source data sources at different magnitudes, for example, in wearable devices or cyber-physical systems, the presence of multiple data streams is commonly seen in our daily lives; 3) often the existing approaches consider osider only homogeneous data modalities (causing interoperability issues). To address such limitations, we propose a deep Multi-Source Adversarial Domain Adaptation (MSADA) framework that considers two labeled source data samples (source domain) and attempts to find annotations for the unlabeled target data samples (target domain). MSADA opportunistically finds the most relevant feature representations from multiple source domains and establishes relevancy to the target sample fea-ture representation by learning the perplexity scores. In MSADA, we minimize the marginal data distribution and ignore the conditional data distribution because of the lack of labeled data in the unlabeled dataset. To complement MSADA, we further study the performance of MSADA in semi-supervised settings where we leverage pseudo-labeling techniques to find annotation of the unlabeled dataset. We find that the MSADA approach achieves 2% and 13% improvement on the OPPORTUNITY and DSADS datasets over the baseline approaches. In SS-MSADA, the pseudo-label data boosts the performance over the unsupervised approach by 2-6% under various domain adaptation scenarios. Intending to develop cross-modal (heterogeneous feature space representation) scalable solutions, our investigation of audio data modality leads us to several challenging scenarios, such as when the audio activities are similar in their execution pattern and can occur in different environments. To tackle these challenges, we propose a contrastive learning-based approach, AcouDL, to exploit the audio reverberation characteristic towards developing a scalable activity recognition approach. In our evaluation of three different datasets, we find that AcouDL performs 1-7% superior in the F1-score compared to the baseline approaches.