UMBC Office for the Vice President of Research & Creative Achievement (ORCA)

Permanent URI for this collectionhttp://hdl.handle.net/11603/14964

The Office of the Vice President for Research and Creative Achievement (ORCA) serves as an advocate for UMBC’s research and creative achievement community, ensures access to key research infrastructure, and supports our faculty, staff and students in their pursuit of research, scholarship and creative achievement across all levels.

Browse

Now showing 1 - 20 of 45

Expanding Polar Science Access and Understanding with Immersive XR
(APL Webinars and Events, 2024-08-21) Tack, Naomi; Holschuh, Nicholas; Sharma, Sharad; Williams, Rebecca M.; Engel, Don
We have developed several immersive fence diagrams which allow scientists to be surrounded by the glacier. Our work focuses on intuitive user interaction, similar to mobile map application controls, and hardware independence allowing greater access among the community. Users are able to navigate through physical motion, panning and adjusting scale allowing many areas of the glacier to be observed in relation to each other. We plan to add tools to allow annotations of the ice sheet layer to be made and visualized in order to trace layers throughout glaciers.
Photogrammetry and VR for Comparing 2D and Immersive Linguistic Data Collection (Student Abstract)
(AAAI, 2023-06-26) Rubinstein, Jacob; Matuszek, Cynthia; Engel, Don
The overarching goal of this work is to enable the collection of language describing a wide variety of objects viewed in virtual reality. We aim to create full 3D models from a small number of ‘keyframe’ images of objects found in the publicly available Grounded Language Dataset (GoLD) using photogrammetry. We will then collect linguistic descriptions by placing our models in virtual reality and having volunteers describe them. To evaluate the impact of virtual reality immersion on linguistic descriptions of the objects, we intend to apply contrastive learning to perform grounded language learning, then compare the descriptions collected from images (in GoLD) versus our models.
Visualizing Earth Science Climate Model and Digital Twin data in XR
(AGU, 2024-12-10) Grubb, Thomas; Kullman, Kaur; Clune, Thomas; Lait, Leslie R.; Zwicker, Matthias; Guimond, Stephen; West, Ruth; Eastman, Roger; Engel, Don; Ames, Troy; Hosler, Jeffrey; Hegde, Srinidhi
Earth Science benefits from a vast treasure trove of in-situ and remotely sensed observational data as well as complex modeling products. Earth Science Digital Twins (ESDT) are an emerging capability that integrates and synthesizes this and other data to help with understanding and forecasting the complex interconnections among Earth systems
Blue Sky: Expert-in-the-Loop Representation Learning Framework for Audio Anti-Spoofing: Multimodal, Multilingual, Multi-speaker, Multi-attack (4M) Scenarios
(SIAM International Conference on Data Mining, 2025) Khanjani, Zahra; Janeja, Vandana; Mallinson, Christine; Purushotham, Sanjay
Audio spoofing has surged with the rise of generative artificial intelligence, posing a serious threat to online communication. Recent studies have shown promising avenues in detecting spoofed audio specifically those that use human expert knowledge in representation learning, but more work is needed to evaluate performance across various realistic scenarios that tend to pose challenges in spoofed audio detection. In this paper, we introduce a comprehensive framework for expert-in-the-loop representation learning for audio anti-spoofing that is robust enough to address four specific challenging scenarios. Multimodal, Multilingual, Multi-speaker, and Multi-attack (4M). Preliminary results demonstrate the framework’s potential effectiveness in audio anti-spoofing.
Artifact: Defining and Analyzing Smart Device Passive Mode
(HAL, 2025-03-17) Badolato, Christian; Kullman, Kaur; Papadakis, Nikolaos; Bhatt, Manav; Bouloukakis, Georgios; Engel, Don; Yus, Roberto
This artifact paper presents a guide for the Smart Home IoT Passive Mode Analysis tool and dataset to perform network traffic analysis (NTA) on smart home IoT devices in passive mode. The repository includes: 1) scripts and configurations for processing network traffic capture files and extracting the relevant information; 2) output data files for the experiments conducted; and 3) a link to the raw network capture dataset. The dataset contains 12GB of passive mode traffic from 32 devices across 3 testbeds; between 71 and 196 hours of traffic is present for each device.
Your smart home exchanged 3M messages: defining and analyzing smart device passive mode
(HAL, 2025-03) Badolato, Christian; Kullman, Kaur; Papadakis, Nikolaos; Bhatt, Manav; Bouloukakis, Georgios; Engel, Don; Yus, Roberto
The constant connectedness of smart home devices and their sensing capabilities pose a unique threat to individuals’ privacy. While users may expect devices to exhibit minimal activity while they are not performing their intended functions, this is not necessarily the case, and traditional idle mode designations are insufficient to address the current landscape of smart home devices. To address this we propose a passive mode designation based on a comprehensive categorization of smart home devices. We then measure the network traffic of thirty-two devices in their respective passive modes. We find that 97% of the devices exhibit near-constant network activity in these modes (exchanging over 3M messages in 24 hours), with many of the devices initiating and responding to LAN communications with other devices, which potentially exposes users to privacy leakages.
Visualizing the Greenland Ice Sheet in VR using Immersive Fence Diagrams
(ACM, 2023-09-10) Tack, Naomi; Williams, Rebecca M.; Holschuh, Nicholas; Sharma, Sharad; Engel, Don
The melting of the ice sheets covering Greenland and Antarctica are primary drivers of sea level rise. Predicting the rate of ice loss depends on modeling the ice dynamics. Ice penetrating radar provides the ability to capture images through the ice sheet, down to the bedrock. Historical environmental and climate perturbations cause small changes to the dielectric constant of ice, which are visually manifested as layers of varying brightness in the radar imagery. To understand how the flow of ice has progressed between neighboring image slices, glaciologists use Fence Diagrams to visualize several cross-sections at once. Here, we describe the immersive virtual reality (VR) fence diagrams we have developed. The goal of our system is to enable glaciologists to make sense of these data and thereby predict future ice loss.
Development and Initial Testing of XR-Based Fence Diagrams for Polar Science
(IEEE, 2023-07) Tack, Naomi; Holschuh, Nicholas; Sharma, Sharad; Williams, Rebecca M.; Engel, Don
Earth’s ice sheets are the largest contributor to sea level rise. For this reason, understanding the flow and topology of ice sheets is crucial for the development of accurate models and predictions. In order to aid in the generation of such models, ice penetrating radar is used to collect images of the ice sheet through both airborne and ground-based platforms. Glaciologists then take these images and visualize them in 3D fence diagrams on a flat 2D screen. We aim to consider the benefits that an XR visualization of these diagrams may provide to enable better data comprehension, annotation, and collaborative work. In this paper, we discuss our initial development and evaluation of such an XR system.
Listening for Expert Identified Linguistic Features: Assessment of Audio Deepfake Discernment among Undergraduate Students
(2024-11-21) Bhalli, Noshaba Nasir; Naqvi, Nehal; Evered, Chloe; Mallinson, Christine; Janeja, Vandana
This paper evaluates the impact of training undergraduate students to improve their audio deepfake discernment ability by listening for expert-defined linguistic features. Such features have been shown to improve performance of AI algorithms; here, we ascertain whether this improvement in AI algorithms also translates to improvement of the perceptual awareness and discernment ability of listeners. With humans as the weakest link in any cybersecurity solution, we propose that listener discernment is a key factor for improving trustworthiness of audio content. In this study we determine whether training that familiarizes listeners with English language variation can improve their abilities to discern audio deepfakes. We focus on undergraduate students, as this demographic group is constantly exposed to social media and the potential for deception and misinformation online. To the best of our knowledge, our work is the first study to uniquely address English audio deepfake discernment through such techniques. Our research goes beyond informational training by introducing targeted linguistic cues to listeners as a deepfake discernment mechanism, via a training module. In a pre-/post- experimental design, we evaluated the impact of the training across 264 students as a representative cross section of all students at the University of Maryland, Baltimore County, and across experimental and control sections. Findings show that the experimental group showed a statistically significant decrease in their unsurety when evaluating audio clips and an improvement in their ability to correctly identify clips they were initially unsure about. While results are promising, future research will explore more robust and comprehensive trainings for greater impact.
Commentary: Bringing people and technology together to combat the threat of deepfakes
(Maryland Matters, 2024-03-25) Mallinson, Christine; Janeja, Vandana
UMBC team is creating and testing short training sessions to help listeners spot common 'tells' of real and fake speech.
ALDAS: Audio-Linguistic Data Augmentation for Spoofed Audio Detection
(2024-10-21) Khanjani, Zahra; Mallinson, Christine; Foulds, James; Janeja, Vandana
Spoofed audio, i.e. audio that is manipulated or AI-generated deepfake audio, is difficult to detect when only using acoustic features. Some recent innovative work involving AI-spoofed audio detection models augmented with phonetic and phonological features of spoken English, manually annotated by experts, led to improved model performance. While this augmented model produced substantial improvements over traditional acoustic features based models, a scalability challenge motivates inquiry into auto labeling of features. In this paper we propose an AI framework, Audio-Linguistic Data Augmentation for Spoofed audio detection (ALDAS), for auto labeling linguistic features. ALDAS is trained on linguistic features selected and extracted by sociolinguistics experts; these auto labeled features are used to evaluate the quality of ALDAS predictions. Findings indicate that while the detection enhancement is not as substantial as when involving the pure ground truth linguistic features, there is improvement in performance while achieving auto labeling. Labels generated by ALDAS are also validated by the sociolinguistics experts.
Toward Transdisciplinary Approaches to Audio Deepfake Discernment
(2024-11-08) Janeja, Vandana; Mallinson, Christine
This perspective calls for scholars across disciplines to address the challenge of audio deepfake detection and discernment through an interdisciplinary lens across Artificial Intelligence methods and linguistics. With an avalanche of tools for the generation of realistic-sounding fake speech on one side, the detection of deepfakes is lagging on the other. Particularly hindering audio deepfake detection is the fact that current AI models lack a full understanding of the inherent variability of language and the complexities and uniqueness of human speech. We see the promising potential in recent transdisciplinary work that incorporates linguistic knowledge into AI approaches to provide pathways for expert-in-the-loop and to move beyond expert agnostic AI-based methods for more robust and comprehensive deepfake detection.
ALDAS: Audio-Linguistic Data Augmentation for Spoofed Audio Detection
(2024-10-21) Khanjani, Zahra; Mallinson, Christine; Foulds, James; Janeja, Vandana
Spoofed audio, i.e. audio that is manipulated or AI-generated deepfake audio, is difficult to detect when only using acoustic features. Some recent innovative work involving AI-spoofed audio detection models augmented with phonetic and phonological features of spoken English, manually annotated by experts, led to improved model performance. While this augmented model produced substantial improvements over traditional acoustic features based models, a scalability challenge motivates inquiry into auto labeling of features. In this paper we propose an AI framework, Audio-Linguistic Data Augmentation for Spoofed audio detection (ALDAS), for auto labeling linguistic features. ALDAS is trained on linguistic features selected and extracted by sociolinguistics experts; these auto labeled features are used to evaluate the quality of ALDAS predictions. Findings indicate that while the detection enhancement is not as substantial as when involving the pure ground truth linguistic features, there is improvement in performance while achieving auto labeling. Labels generated by ALDAS are also validated by the sociolinguistics experts.
Commentary: Bringing people and technology together to combat the threat of deepfakes
(Maryland Matters, 2024-03-25) Mallinson, Christine; Janeja, Vandana
UMBC team is creating and testing short training sessions to help listeners spot common 'tells' of real and fake speech.
The Use of Guided Reflections in Learning Proof Writing
(MDPI, 2024-10-04) Hoffman, Kathleen; Williams, Tory; Kephart, Kerrie
We investigated written self-reflections in an undergraduate proof-writing course designed to mitigate the difficulty of a subsequent introductory analysis course. Students wrote weekly self-reflections guided by mechanical, structural, creative, and critical thinking modalities. Our research was guided by three research questions focused on the impact of student self-reflections on student metacognition and performance in the interventional and follow-up class. To address these questions, we categorized the quality of the students’ reflections and calculated their average course grades within each category in the proof-writing, the prerequisite, and the introductory analysis courses. The results demonstrated that writing high-quality self-reflections was a statistically significant predictor of earning higher average course grades in the proof-writing course and the analysis course, but not in the prerequisite course. Convergence over the semester of the students’ self-evaluations toward an experts’ scorings on a modality rubric indicates that students improve in their understanding of the modalities. The repeated writing of guided self-reflections using the framework of the modalities seems to support growth in the students’ awareness of their proof-writing abilities.
Investigating Causal Cues: Strengthening Spoofed Audio Detection with Human-Discernible Linguistic Features
(2024-09-09) Khanjani, Zahra; Ale, Tolulope; Wang, Jianwu; Davis, Lavon; Mallinson, Christine; Janeja, Vandana
Several types of spoofed audio, such as mimicry, replay attacks, and deepfakes, have created societal challenges to information integrity. Recently, researchers have worked with sociolinguistics experts to label spoofed audio samples with Expert Defined Linguistic Features (EDLFs) that can be discerned by the human ear: pitch, pause, word-initial and word-final release bursts of consonant stops, audible intake or outtake of breath, and overall audio quality. It is established that there is an improvement in several deepfake detection algorithms when they augmented the traditional and common features of audio data with these EDLFs. In this paper, using a hybrid dataset comprised of multiple types of spoofed audio augmented with sociolinguistic annotations, we investigate causal discovery and inferences between the discernible linguistic features and the label in the audio clips, comparing the findings of the causal models with the expert ground truth validation labeling process. Our findings suggest that the causal models indicate the utility of incorporating linguistic features to help discern spoofed audio, as well as the overall need and opportunity to incorporate human knowledge into models and techniques for strengthening AI models. The causal discovery and inference can be used as a foundation of training humans to discern spoofed audio as well as automating EDLFs labeling for the purpose of performance improvement of the common AI-based spoofed audio detectors.
Disciplinary Differences in STEM Faculty and Student Use of Learning Objectives: Implications for Teaching and Learning
(Taylor & Francis, 2024-07-22) Leupen, Sarah; Williams, Tory; Hodges, Linda C.; Ott, Laura E.; Anderson, Eric C.; Cui, Lili; Nanes, Kalman M.; Perks, H. Mark; Wagner, Cynthia R.
Using learning objectives to guide course design is often considered an educational best practice, but little research exists that explores how students use them over time and across courses. We surveyed students on their use and perceived value of learning objectives as the semester progressed across four science, technology, engineering, and mathematics (STEM) disciplines, examined students’ ability to match exam questions with learning objectives, and analyzed how their course performance related to these qualities. We also gathered instructors’ information on their implementation of learning objectives in these courses. We identified distinct disciplinary differences both in students’ use and perceived benefit of learning objectives and in instructors’ implementation of them. Students in less quantitatively focused courses, i.e., biology and organic chemistry, reported valuing and using learning objectives more than students in more quantitatively focused math and physics courses. Students’ ability to match learning objectives with exam questions, however, positively correlated with exam score and final course grade in all our study courses. Our results have implications for considering disciplinary practices for use of learning objectives as instructors design and implement courses, educational researchers plan studies, and assessment specialists formulate institutional assessment plans.
Metrics for the Quality and Consistency of Ice Layer Annotations
(IEEE, 2023-07) Tack, Naomi; Tama, Bayu Adhi; Jebeli, Atefeh; Janeja, Vandana; Engel, Don; Williams, Rebecca M.
Ice layers in glaciers, such as those covering Greenland and Antarctica, are deformed over time. The deformations of these layers provide a record of climate history and are useful in predicting future ice flow and ice loss. Cross sectional images of the ice can be captured by airborne radar and layers in the images then annotated by glaciologists. Recent advances in semi-automated and automated annotation allow for significantly more annotations, but the validity of these annotations is difficult to determine because ground-truth (GT) data is scarce. In this paper, we (1) propose GT-dependent and GT-independent metrics for layer annotations and (2) present results from our implementation and initial testing of GT-independent metrics, such as layer breakpoints, local layer density, spatial frequency, and layer orientation agreement.
A place for (socio)linguistics in audio deepfake detection and discernment: Opportunities for convergence and interdisciplinary collaboration
(Wiley, 2024-07-09) Mallinson, Christine; Janeja, Vandana; Evered, Chloe; Khanjani, Zahra; Davis, Lavon; Bhalli, Noshaba Nasir; Nwosu, Kifekachukwu
Deepfakes, particularly audio deepfakes, have become pervasive and pose unique, ever-changing threats to society. This paper reviews the current research landscape on audio deepfakes. We assert that limitations of existing approaches to deepfake detection and discernment are areas where (socio)linguists can directly contribute to helping address the societal challenge of audio deepfakes. In particular, incorporating expert knowledge and developing techniques that everyday listeners can use to avoid deception are promising pathways for (socio)linguistics. Further opportunities exist for developing benevolent applications of this technology through generative AI methods as well.
UMBC Scientists And Engineers Celebrate Launch Of HARP2 Instrument On NASA's PACE Mission
(UMBC News, 2024-02-16) Wainscott-Sargent, Anne
After over a decade of concerted effort, full of setbacks and recoveries, UMBC's HARP team celebrated as the instrument they designed and built launched on PACE, a major NASA mission set to study Earth's atmosphere and oceans.

Browse

Recent Submissions