Hossain, JummanDey, EmonChugh, SnehalrajAhmed, MasudAnwar,Mohammad SaeidFaridee, Abu Zaher MdHoppes, JasonTrout, TheronBasak, AnjonChowdhury, RafidhMistry, RishabhKim, HyunFreeman, JadeSuri, NiranjanRaglin, AdrienneBusart, CarlGregory, TimothyRavi, AnuradhaRoy, Nirmalya2024-12-112024-12-112024-10-22https://doi.org/10.48550/arXiv.2410.16686http://hdl.handle.net/11603/37081The increasing deployment of autonomous systems in complex environments necessitates efficient communication and task completion among multiple agents. This paper presents SERN (Simulation-Enhanced Realistic Navigation), a novel framework integrating virtual and physical environments for real-time collaborative decision-making in multi-robot systems. SERN addresses key challenges in asset deployment and coordination through a bi-directional communication framework using the AuroraXR ROS Bridge. Our approach advances the SOTA through accurate real-world representation in virtual environments using Unity high-fidelity simulator; synchronization of physical and virtual robot movements; efficient ROS data distribution between remote locations; and integration of SOTA semantic segmentation for enhanced environmental perception. Our evaluations show a 15% to 24% improvement in latency and up to a 15% increase in processing efficiency compared to traditional ROS setups. Real-world and virtual simulation experiments with multiple robots demonstrate synchronization accuracy, achieving less than 5 cm positional error and under 2-degree rotational error. These results highlight SERN's potential to enhance situational awareness and multi-agent coordination in diverse, contested environments.8 pagesen-USThis work was written as part of one of the author's official duties as an Employee of the United States Government and is therefore a work of the United States Government. In accordance with 17 U.S.C. 105, no copyright protection is available for such works under U.S. Law.Public Domainhttps://creativecommons.org/publicdomain/mark/1.0/UMBC Mobile, Pervasive and Sensor Computing Lab (MPSC Lab)Computer Science - Multiagent SystemsComputer Science - RoboticsText