Blind Source Separation for Multimodal Fusion of Medical Imaging Data

Author/Creator ORCID




Computer Science and Electrical Engineering


Engineering, Electrical

Citation of Original Publication


This item may be protected under Title 17 of the U.S. Copyright Law. It is made available by UMBC for non-commercial research and education. For permission to publish or reproduce, please see or contact Special Collections at speccoll(at)
Distribution Rights granted to UMBC by the author.


Due to the ability of different sensors to provide complementary views of complicated systems, the collection of data from multiple sources has become common, particularly in neurological studies. Therefore, full utilization of the common information forms the fundamental goal of performing a joint analysis on this data. However, since little is known about the relationships among the datasets, it is important to minimize the underlying assumptions placed on the data. Because of this fact and their ability to treat separate datasets in a fully symmetric manner, multivariate data-driven methods are the main choice for the fusion of multiple sets of neurological data. However, different methods rely on different generative models, meaning that many considerations must be taken into account before an individual method can be applied to a new problem. This motivates an investigation of how the assumptions of different methods manifest in the resulting decomposition, the development of techniques to assess the contribution of different datasets to the analysis, and the development of a way to unambiguously assess the relative performance of different methods on real data. In this dissertations, we approach these issues from multiple directions. We introduce a technique called principal component analysis and canonical correlation analysis (PCA-CCA) to determine the similarity or links between different neuroimaging datasets. Through both simulations and application to brain imaging data, namely, functional magnetic resonance imaging (fMRI) data, structural MRI (sMRI) data, and electroencephalogram (EEG) data from patients with schizophrenia and healthy controls, we show the desirable performance of the proposed technique. We also use this unique set of imaging data in order to test the robustness of different data-driven fusion methods to deviations from their assumptions and to assess the effects of bringing each dataset into the analysis. We propose a classification rate-based procedure to quantify the performance of different fusion methods on real fMRI data extracted during the performance of different tasks and demonstrate how this method can be used to determine the ''value added" by each dataset to the analysis. Finally, we introduce a novel visualization technique to highlight the changes in the brain regions that discriminate between patients with schizophrenia and healthy controls through the use of different fusion techniques. We find that the methods that we develop in this dissertations provide a useful framework for investigating the interactions of different datasets within a fusion analysis.