Integration of Reinforcement Learning and Unreal Engine for Enemy Containment via Autonomous Swarms

Abstract

Maritime remote sensing (MRS) is a multi-disciplinary and multi-physics field at the intersection of naval hydrodynamics, physical oceanography, overhead platforms, and electro-optical sensors. One proposed improvement to MRS information gathering and operations is the use of swarms of autonomous surface, aerial, and/or undersea vehicles as a multi-agent system (MAS) to automate data collection, data processing, and situational awareness. Here, we explore the design of an autonomous multi-agent system with the objective of containing a target object, i.e., surrounding the object in a loosely defined shape. The agents make decisions using reinforcement learning by way of a Markov decision process. Our current proof-of-concepts are modeled using Python-based 2D simulation environments which contain our agents and target used for prototyping and testing various reward functions.. However, we have built an infrastructure to port the simulation environments to Unreal Engine 4 for increased fidelity. In the current modeled scenario, each agent's decisions are based on global positional knowledge of each entity in the environment. Future iterations are planned to feature agent decision making based on a high-fidelity communication protocol and inputs from integrated sensors.