Assessing Pediatric Cognitive Development via Multisensory Brain Imaging Analysis

Abstract

Adolescence is a special period between childhood and adulthood and constitutes a critical developmental stage for humans. During adolescence, the brain processes various stimuli to form a complete view of the world. This study highlights the critical role of multisensory integration, where the brain processes multiple senses together rather than focusing on just one sensory modality at a time. Brain imaging modalities such as magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) can be utilized to gain insights into the non-additive effects of multisensory integration by fusing data across different sensory stimuli in both time and space. While MEG and fMRI are powerful tools, traditional approaches to combining data from these modalities often ignore their multisensory aspect, focusing instead on single tasks. To leverage their complementarity, we introduce a multitask learning multimodal data fusion framework for joint learning of multisensory brain developmental patterns from MEG and fMRI data through a novel application of coupled canonical polyadic tensor decomposition. The multitask learning paradigm performs multimodal fusion from multiple sensory stimuli using multitask coupled tensor-tensor factorization (MCTTF). We demonstrate that multitask multimodal fusion of MEG and fMRI data can identify unique brain components, demonstrating a higher grouplevel multisensory integration effect.