Browsing by Subject "human activity recognition"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Building Robust Human Activity Recognition Models from Unlabeled Data(2022-01-01) Faridee, Abu Zaher Md; Roy, Nirmalya; Information Systems; Information SystemsMachine learning-driven wearable sensor-based human activity recognition (HAR) systems have experienced meteoric popularity in recent years in healthcare, entertainment, and physical fitness applications, but their large-scale adoption has been hampered by several open challenges. The availability of rapidly evolving consumer-grade wearable devices (smart-watch, smart-ring, ear-worns) and the existence of substantial variability in the activities performed by a large number of users � each with their own personal style and demographic variations and the wearables potentially placed in different body positions introduce significant domain and category shifts. The cost-prohibitive nature of developing a large corpus of annotated samples to cover all these heterogeneities is a major hindrance to the development and adoption of scalable supervised HAR models. In response, the recent machine learning literature has increasingly relied on discovering salient features from unlabeled samples. However, these models still impart some restrictions on the model architecture (i.e., inability to handle simultaneous heterogeneities, the requirement of labeled samples or synchronized data collection with multiple sensors, and lack of interpretability). Moreover, their performance still lags behind their supervised counterparts, hindering real-world adoption. In this thesis, we focus on building scalable machine learning models for HAR that are robust against domain shifts with minimal-to-no extra-label information and discover the optimum transferability of the representations between the domains. To that end, we propose a number of deep self-supervised, unsupervised, adversarial representation learning and learnable data augmentation techniques. We first present, AugToAct, a self-supervised representation learning method that utilizes random data transformation with a reconstruction loss to automatically learn salient features from unlabeled samples and retains over 80% F1 score with only 6% labeled samples. We then extend this self-supervised module in a cross-user semi-supervised domain adaptation setup, where it outperforms most state-of-the-art models by a 5% F1 score. In our next work StranGAN, we propose a novel interpretable unsupervised domain adaptation method by adversarially learning a set of affine transformations to align the raw data distributions of source and target domain unlabeled samples without (a) needing to modify the source classifier, (b) having no access to synchronized source and target labeled samples while outperforming state-of-the-art by 5% F1 score. Finally, we present CoDEm, which exploits the domain label meta-data (subjects' gender, sensor position, etc) to learn a set of domain embeddings to capture the salient features pertaining to the underlying heterogeneity. CoDEm provides up to 9.5% improved F1 performance compared to several multi-task learning setups in three public datasets by utilizing these domain embeddings with a novel residual attention mechanism without any loss balancing hyper-parameter search.Item LASO: Exploiting Locomotive and Acoustic Signatures over the Edge to Annotate IMU Data for Human Activity Recognition(ACM) Chatterjee, Soumyajit; Chakma, Avijoy; Gangopadhyay, Aryya; Roy, Nirmalya; Mitra, Bivas; Chakraborty, SandipAnnotated IMU sensor data from smart devices and wearables are essential for developing supervised models for fine-grained human activity recognition, albeit generating sufficient annotated data for diverse human activities under different environments is challenging. Existing approaches primarily use human-in-the-loop based techniques, including active learning; however, they are tedious, costly, and time-consuming. Leveraging the availability of acoustic data from embedded microphones over the data collection devices, in this paper, we propose LASO, a multimodal approach for automated data annotation from acoustic and locomotive information. LASO works over the edge device itself, ensuring that only the annotated IMU data is collected, discarding the acoustic data from the device itself, hence preserving the audio-privacy of the user. In the absence of any pre-existing labeling information, such an auto-annotation is challenging as the IMU data needs to be sessionized for different time-scaled activities in a completely unsupervised manner. We use a change-point detection technique while synchronizing the locomotive information from the IMU data with the acoustic data, and then use pre-trained audio-based activity recognition models for labeling the IMU data while handling the acoustic noises. LASO efficiently annotates IMU data, without any explicit human intervention, with a mean accuracy of $0.93$ ($\pm 0.04$) and $0.78$ ($\pm 0.05$) for two different real-life datasets from workshop and kitchen environments, respectively.