UMBC Office for the Vice President of Research & Creative Achievement (ORCA)

Permanent URI for this collectionhttp://hdl.handle.net/11603/14964

The Office of the Vice President for Research and Creative Achievement (ORCA) serves as an advocate for UMBC’s research and creative achievement community, ensures access to key research infrastructure, and supports our faculty, staff and students in their pursuit of research, scholarship and creative achievement across all levels.

Browse

Now showing 1 - 20 of 34

Toward Transdisciplinary Approaches to Audio Deepfake Discernment
(2024-11-08) Janeja, Vandana; Mallinson, Christine
This perspective calls for scholars across disciplines to address the challenge of audio deepfake detection and discernment through an interdisciplinary lens across Artificial Intelligence methods and linguistics. With an avalanche of tools for the generation of realistic-sounding fake speech on one side, the detection of deepfakes is lagging on the other. Particularly hindering audio deepfake detection is the fact that current AI models lack a full understanding of the inherent variability of language and the complexities and uniqueness of human speech. We see the promising potential in recent transdisciplinary work that incorporates linguistic knowledge into AI approaches to provide pathways for expert-in-the-loop and to move beyond expert agnostic AI-based methods for more robust and comprehensive deepfake detection.
ALDAS: Audio-Linguistic Data Augmentation for Spoofed Audio Detection
(2024-10-21) Khanjani, Zahra; Mallinson, Christine; Foulds, James; Janeja, Vandana
Spoofed audio, i.e. audio that is manipulated or AI-generated deepfake audio, is difficult to detect when only using acoustic features. Some recent innovative work involving AI-spoofed audio detection models augmented with phonetic and phonological features of spoken English, manually annotated by experts, led to improved model performance. While this augmented model produced substantial improvements over traditional acoustic features based models, a scalability challenge motivates inquiry into auto labeling of features. In this paper we propose an AI framework, Audio-Linguistic Data Augmentation for Spoofed audio detection (ALDAS), for auto labeling linguistic features. ALDAS is trained on linguistic features selected and extracted by sociolinguistics experts; these auto labeled features are used to evaluate the quality of ALDAS predictions. Findings indicate that while the detection enhancement is not as substantial as when involving the pure ground truth linguistic features, there is improvement in performance while achieving auto labeling. Labels generated by ALDAS are also validated by the sociolinguistics experts.
Commentary: Bringing people and technology together to combat the threat of deepfakes
(Maryland Matters, 2024-03-25) Mallinson, Christine; Janeja, Vandana
UMBC team is creating and testing short training sessions to help listeners spot common 'tells' of real and fake speech.
The Use of Guided Reflections in Learning Proof Writing
(MDPI, 2024-10-04) Hoffman, Kathleen; Williams, Tory; Kephart, Kerrie
We investigated written self-reflections in an undergraduate proof-writing course designed to mitigate the difficulty of a subsequent introductory analysis course. Students wrote weekly self-reflections guided by mechanical, structural, creative, and critical thinking modalities. Our research was guided by three research questions focused on the impact of student self-reflections on student metacognition and performance in the interventional and follow-up class. To address these questions, we categorized the quality of the students’ reflections and calculated their average course grades within each category in the proof-writing, the prerequisite, and the introductory analysis courses. The results demonstrated that writing high-quality self-reflections was a statistically significant predictor of earning higher average course grades in the proof-writing course and the analysis course, but not in the prerequisite course. Convergence over the semester of the students’ self-evaluations toward an experts’ scorings on a modality rubric indicates that students improve in their understanding of the modalities. The repeated writing of guided self-reflections using the framework of the modalities seems to support growth in the students’ awareness of their proof-writing abilities.
Investigating Causal Cues: Strengthening Spoofed Audio Detection with Human-Discernible Linguistic Features
(2024-09-09) Khanjani, Zahra; Ale, Tolulope; Wang, Jianwu; Davis, Lavon; Mallinson, Christine; Janeja, Vandana
Several types of spoofed audio, such as mimicry, replay attacks, and deepfakes, have created societal challenges to information integrity. Recently, researchers have worked with sociolinguistics experts to label spoofed audio samples with Expert Defined Linguistic Features (EDLFs) that can be discerned by the human ear: pitch, pause, word-initial and word-final release bursts of consonant stops, audible intake or outtake of breath, and overall audio quality. It is established that there is an improvement in several deepfake detection algorithms when they augmented the traditional and common features of audio data with these EDLFs. In this paper, using a hybrid dataset comprised of multiple types of spoofed audio augmented with sociolinguistic annotations, we investigate causal discovery and inferences between the discernible linguistic features and the label in the audio clips, comparing the findings of the causal models with the expert ground truth validation labeling process. Our findings suggest that the causal models indicate the utility of incorporating linguistic features to help discern spoofed audio, as well as the overall need and opportunity to incorporate human knowledge into models and techniques for strengthening AI models. The causal discovery and inference can be used as a foundation of training humans to discern spoofed audio as well as automating EDLFs labeling for the purpose of performance improvement of the common AI-based spoofed audio detectors.
Disciplinary Differences in STEM Faculty and Student Use of Learning Objectives: Implications for Teaching and Learning
(Taylor & Francis, 2024-07-22) Leupen, Sarah; Williams, Tory; Hodges, Linda C.; Ott, Laura E.; Anderson, Eric C.; Cui, Lili; Nanes, Kalman M.; Perks, H. Mark; Wagner, Cynthia R.
Using learning objectives to guide course design is often considered an educational best practice, but little research exists that explores how students use them over time and across courses. We surveyed students on their use and perceived value of learning objectives as the semester progressed across four science, technology, engineering, and mathematics (STEM) disciplines, examined students’ ability to match exam questions with learning objectives, and analyzed how their course performance related to these qualities. We also gathered instructors’ information on their implementation of learning objectives in these courses. We identified distinct disciplinary differences both in students’ use and perceived benefit of learning objectives and in instructors’ implementation of them. Students in less quantitatively focused courses, i.e., biology and organic chemistry, reported valuing and using learning objectives more than students in more quantitatively focused math and physics courses. Students’ ability to match learning objectives with exam questions, however, positively correlated with exam score and final course grade in all our study courses. Our results have implications for considering disciplinary practices for use of learning objectives as instructors design and implement courses, educational researchers plan studies, and assessment specialists formulate institutional assessment plans.
Metrics for the Quality and Consistency of Ice Layer Annotations
(IEEE, 2023-07) Tack, Naomi; Tama, Bayu Adhi; Jebeli, Atefeh; Janeja, Vandana; Engel, Don; Williams, Rebecca
Ice layers in glaciers, such as those covering Greenland and Antarctica, are deformed over time. The deformations of these layers provide a record of climate history and are useful in predicting future ice flow and ice loss. Cross sectional images of the ice can be captured by airborne radar and layers in the images then annotated by glaciologists. Recent advances in semi-automated and automated annotation allow for significantly more annotations, but the validity of these annotations is difficult to determine because ground-truth (GT) data is scarce. In this paper, we (1) propose GT-dependent and GT-independent metrics for layer annotations and (2) present results from our implementation and initial testing of GT-independent metrics, such as layer breakpoints, local layer density, spatial frequency, and layer orientation agreement.
A place for (socio)linguistics in audio deepfake detection and discernment: Opportunities for convergence and interdisciplinary collaboration
(Wiley, 2024-07-09) Mallinson, Christine; Janeja, Vandana; Evered, Chloe; Khanjani, Zahra; Davis, Lavon; Bhalli, Noshaba Nasir; Nwosu, Kifekachukwu
Deepfakes, particularly audio deepfakes, have become pervasive and pose unique, ever-changing threats to society. This paper reviews the current research landscape on audio deepfakes. We assert that limitations of existing approaches to deepfake detection and discernment are areas where (socio)linguists can directly contribute to helping address the societal challenge of audio deepfakes. In particular, incorporating expert knowledge and developing techniques that everyday listeners can use to avoid deception are promising pathways for (socio)linguistics. Further opportunities exist for developing benevolent applications of this technology through generative AI methods as well.
UMBC Scientists And Engineers Celebrate Launch Of HARP2 Instrument On NASA's PACE Mission
(UMBC News, 2024-02-16) Wainscott-Sargent, Anne
After over a decade of concerted effort, full of setbacks and recoveries, UMBC's HARP team celebrated as the instrument they designed and built launched on PACE, a major NASA mission set to study Earth's atmosphere and oceans.
Drones, Phones, and Stones: Visualization for Collaborative Digitization of Historic Cemeteries
(ACM, 2023-09-10) Engel, Don; Rubinstein, Jacob
Historic cemeteries face ravages of time, weather, and vandalism. We first captured hundreds of images for each of several unindexed historic cemeteries. We then used a high performance computing system to orthorectify these images into ultraresolution (billions of pixels), georeferenced, tiled images. Here, we describe the interactive visualization we developed using these tiled images to enable volunteers to collaboratively map graves and identify burials. Our ongoing work tests the capacity of this visualization to enable coordination between volunteers with distinct and often asynchronous assignments, including photography, mapping, research, and restoration.
Surveying the landscape of college teaching about African American Language
(Elsevier, 2023-06-13) Sedlacek, Quentin C.; Hudley, Anne H. Charity; Mallinson, Christine
College courses are an important forum for combating the stigmatization of African American Language (AAL). However, there is no comprehensive data regarding where, how, and by whom AAL content is taught. Understanding the landscape of college teaching about AAL could help identify challenges faced by instructors who teach this content, as well as policies or practices that could help support these instructors. We surveyed college instructors (N = 149) in multiple disciplines (primarily Linguistics, Education, English, and Communication Sciences) who teach courses with AAL content. We found patterns in the sources of support and levels of resistance instructors reported. Instructors also expressed varied levels of knowledge and confidence related to teaching about African American Language and Culture. Many of these patterns were correlated with instructors’ racialized identities and language backgrounds. We discuss implications for professional organizations, university department leaders, and instructors who teach AAL content.
A Comparative Analysis of VR-Based and Real-World Human-Robot Collaboration for Small-Scale Joining
(IEEE, 2023-05-01) Higgins, Padraig; Barron, Ryan; Engel, Don; Matuszek, Cynthia
While VR as an interface for the teleoperation of robots has been well-studied in recent years, VR can also be used to advance our understanding of in-person human-robot interaction (HRI) by sim-ulating such interactions more repeatably and affordably than real-world studies. A few platforms now exist for studying human-robot interaction in VR, but little of this work has involved the study of the realism of specific, typical in-person HRI tasks. To evaluate this realism, we conduct a user study consisting of a collaborative assembly task where a robot and human work together to build a simple electrical circuit. We present a comparison of the task performed in the real world versus with virtual robot in VR. We discuss difficulties encountered and draw conclusions about what characteristics a virtual environment should have in order to support physical human-robot interactions.
Using Moffat Profiles to Register Astronomical Images
(Springer, 2023-02-15) Schuckman, Mason; Prouty, Roy; Chapman, David; Engel, Don
The accurate registration of astronomical images without a world coordinate system or authoritative catalog is useful for visually enhancing the spatial resolution of multiple images containing the same target. Increasing the resolution of images through super-resolution (SR) techniques can improve the performance of commodity optical hardware, allowing more science to be done with cheaper equipment. Many SR techniques rely on the accurate registration of input images, which is why this work is focused on accurate star finding and registration. In this work, synthetic star field frames are used to explore techniques involving star detection, matching, and transform-fitting. Using Moffat stellar profiles for stars, non-maximal suppression for control-point finding, and gradient descent for point finding optimization, we are able to obtain more accurate transformation parameters than that provided other modern algorithms, e.g., AstroAlign. To validate that we do not over-fit our method to our synthetic images, we use real telescope images and attempt to recover the transformation parameters.
Integration of Reinforcement Learning and Unreal Engine for Enemy Containment via Autonomous Swarms
(AIAA, 2023-01-19) Peterson, David; Andrades, Beyonce; Lizarazu-Ampuero, Kevin; Deshmukh, Jai; Stapor, Thomas; Destaffan, Will; Engel, Don; Krometis, Justin; Kauffman, Justin A.
Maritime remote sensing (MRS) is a multi-disciplinary and multi-physics field at the intersection of naval hydrodynamics, physical oceanography, overhead platforms, and electro-optical sensors. One proposed improvement to MRS information gathering and operations is the use of swarms of autonomous surface, aerial, and/or undersea vehicles as a multi-agent system (MAS) to automate data collection, data processing, and situational awareness. Here, we explore the design of an autonomous multi-agent system with the objective of containing a target object, i.e., surrounding the object in a loosely defined shape. The agents make decisions using reinforcement learning by way of a Markov decision process. Our current proof-of-concepts are modeled using Python-based 2D simulation environments which contain our agents and target used for prototyping and testing various reward functions.. However, we have built an infrastructure to port the simulation environments to Unreal Engine 4 for increased fidelity. In the current modeled scenario, each agent's decisions are based on global positional knowledge of each entity in the environment. Future iterations are planned to feature agent decision making based on a high-fidelity communication protocol and inputs from integrated sensors.
Drones, Phones, and Stones: Initial Testing of a Role-Based, Computer-Supported Approach to Collaborative Cemetery Indexing
(ACM, 2023-01-08) Rubinstein, Jacob; Engel, Don
After project organizers used a drone to create a high-resolution aerial map of a historic cemetery, sets of volunteers served in four distinct, interdependent roles to photograph, research, restore, and geospatially index the cemetery’s many unrecorded interments. Here, we share observations from our initial tests of this approach, which included a visit of volunteers for synchronous on-site work. We explore the differences in volunteer behavior and performance relative to organizer expectations and how the interdependence of the four roles is impacted by these differences.
Mobile augmented reality system for object detection, alert, and safety
(Society for Imaging Science and Technology, 2023-01) Sharma, Sharad; Engel, Don
The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in daily life. Mobile devices and applications provide many benefits for users, perhaps most significantly is the increased access to point-of-use tools, navigation, and alert systems. This paper presents a prototype of a cross-platform mobile augmented reality (AR) system with the core purpose of finding a better means to keep the campus community secure and connected. The mobile AR System consists of four core functionalities – an events system, a policing system, a directory system, and a notification system. The events system keeps the community up-to-date on current events that are happening or will be happening on campus. The policing system allows the community to stay in arms reach of campus resources that will allow them to stay secure. The directory system serves as a one-stop-shop for campus resources, ensuring that staff, faculty, and students will have a convenient and efficient means of accessing pertinent information on the campus departments. The mobile augmented reality system includes integrated guided navigation system that users can use to get directions to various destinations on campus. The various destinations are different buildings and departments on campus. This mobile augmented reality application will assist the students and visitors on campus to efficiently navigate the campus as well as send alert and notifications in case of emergencies. This will allow campus police to respond to the emergencies in a quick and timely manner. The mobile AR system was designed using Unity Game Engine and Vuforia Engine for object detection and classification. Google Map API was integrated for GPS integration in order to provide location-based services. Our contribution lies in our approach to create a user specific customizable navigational and alert system in order to improve the safety of the users at their workplace. Specifically, the paper describes the design and implementation of the proposed mobile AR system and reports the results of the pilot study conducted to evaluate their perceived ease-of-use, and usability.
A Spoken Language Dataset of Descriptions for Speech-Based Grounded Language Learning
(2021-07-29) Kebe, Gaoussou Youssouf; Higgins, Padraig; Jenkins, Patrick; Darvish, Kasra; Sachdeva, Rishabh; Barron, Ryan; Winder, John; Engel, Don ; Raff, Edward; Ferraro, Francis; Matuszek, Cynthia
Grounded language acquisition is a major area of research combining aspects of natural language processing, computer vision, and signal processing, compounded by domain issues requiring sample efficiency and other deployment constraints. In this work, we present a multimodal dataset of RGB+depth objects with spoken as well as textual descriptions. We analyze the differences between the two types of descriptive language and our experiments demonstrate that the different modalities affect learning. This will enable researchers studying the intersection of robotics, NLP, and HCI to better investigate how the multiple modalities of image, depth, text, speech, and transcription interact, as well as how differences in the vernacular of these modalities impact results.
Lessons From A Small-Scale Robot Joining Experiment in VR
(2023-03) Higgins, Padraig; Barron, Ryan; Engel, Don; Matuszek, Cynthia
In this paper, we present a shared manipulation task performed both in virtual reality with a simulated robot and in the real world with a physical robot. A collaborative assembly task where the human and robot work together to construct as simple electrical circuit was chosen. While there are platforms available for conducting human robot interactions using virtual reality, there has not been significant work investigating how it can influence human perception of tasks that are typically done in person. We present an overview of the simulation environment used, describe the paired experiment being performed, and finally enumerate a set of design desiderata to be considered when conducting sim2real experiment involving humans in a virtual setting.
User Interactions in Virtual Data Explorer
(Springer, 2022-06-16) Kullman, Kaur; Engel, Don
Cybersecurity practitioners face the challenge of monitoring complex and large datasets. These could be visualized as time-varying node-link graphs, but would still have complex topologies and very high rates of change in the attributes of their links (representing network activity). It is natural, then, that the needs of the cybersecurity domain have driven many innovations in 2D visualization and related computer-assisted decision making. Here, we discuss the lessons learned while implementing user interactions for Virtual Data Explorer (VDE), a novel system for immersive visualization (both in Mixed and Virtual Reality) of complex time-varying graphs. VDE can be used with any dataset to render its topological layout and overlay that with time-varying graph; VDE was inspired by the needs of cybersecurity professionals engaged in computer network defense (CND). Immersive data visualization using VDE enables intuitive semantic zooming, where the semantic zoom levels are determined by the spatial position of the headset, the spatial position of handheld controllers, and user interactions (UIa) with those controllers. This spatially driven semantic zooming is quite different from most other network visualizations which have been attempted with time-varying graphs of the sort needed for CND, presenting a broad design space to be evaluated for overall user experience (UX) optimization. In this paper, we discuss these design choices, as informed by CND experts, with a particular focus on network topology abstraction with graph visualization, semantic zooming on increasing levels of network detail, and semantic zooming to show increasing levels of detail with textual labels.
Interactive Stereoscopically Perceivable Multidimensional Data Visualizations for Cybersecurity
(Journal of Defence & Security Technologies, 2021-12-12) Kullman, Kaur; Engel, Don
Interactive Data Visualizations (IDV) can be useful for cybersecurity subject matter experts (CSMEs) while they are exploring new data or investigating familiar datasets for anomalies, correlating events, etc. For an IDV to be useful to a CSME, interaction with that visualization should be simple and intuitive (free of additional mental tasks) and the visualization’s layout must map to a CSME’s understanding. While CSMEs may learn to interpret visualizations created by others, they should be encouraged to visualize their datasets in ways that best reflect their own ways of thinking. Developing their own visual schemes makes optimal use of both the data analysis tools and human visual cognition. In this article, we focus on a currently available interactive stereoscopically perceivable multidimensional data visualization solution, as such tools could provide CSMEs with better perception of their data compared to interpreting IDV on flat media (whether visualized as 2D or 3D structures).

Browse

Recent Submissions