Decoding concealed information using multimodal neurophysiological signals

Abstract

Detecting concealed information is a critical challenge in forensic investigations, security screening, and cognitive neuroscience. Conventional approaches using the Concealed Information Test primarily rely on binary classification, distinguishing between recognized (concealed) and unrecognized (neutral) stimuli. This limits interpretability and fails to reveal the nature of the concealed knowledge. In this study, we present a novel multimodal framework that moves beyond binary detection by decoding the category of concealed information into object, person, or place based on neurophysiological signals recorded during a concealed information test. High density electroencephalography and physiological signals, including skin temperature, galvanic skin response, and photoplethysmogram, were simultaneously recorded from ten subjects as they viewed visual stimuli representing these three categories. Temporal and spectral features were extracted from both modalities, followed by machine learning based multimodal fusion for classification. The proposed framework achieved an overall accuracy of 94.2%, significantly outperforming unimodal EEG (73%) and physiological (54.2%) baselines. Further analysis showed that similar decoding performance can be achieved using as few as eight strategically selected EEG electrodes, supporting the feasibility of lightweight, wearable implementations. The most informative electrodes were located over prefrontal and frontocentral regions, aligning with cognitive processes related to attention, recognition, and deception. These findings demonstrate that neurophysiological signals enables not only the detection of concealed knowledge but also the identification of the type of hidden information. The integration of EEG and physiological signals enhances both sensitivity and interpretability by capturing complementary aspects of cognitive and affective processing during recognition. By enhancing the CIT paradigm from binary recognition toward semantic decoding, this study advances the development of interpretable deception detection systems and bridges laboratory neuroscience with real world forensic applications.