Piplai, AritranAnoruo, MikeFasaye, KayodeJoshi, AnupamFinin, TimRidley, Ahmad2022-12-202022-12-202023-03-23A. Piplai, M. Anoruo, K. Fasaye, A. Joshi, T. Finin and A. Ridley, "Knowledge Guided Two-player Reinforcement Learning for Cyber Attacks and Defenses," 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA), Nassau, Bahamas, 2022, pp. 1342-1349, doi: 10.1109/ICMLA55696.2022.00213.http://hdl.handle.net/11603/26478https://doi.org/10.1109/ICMLA55696.2022.00213International Conference on Machine Learning and ApplicationsCyber defense exercises are an important avenue to understand the technical capacity of organizations when faced with cyber-threats. Information derived from these exercises often leads to finding unseen methods to exploit vulnerabilities in an organization. These often lead to better defense mechanisms that can counter previously unknown exploits. With recent developments in cyber battle simulation platforms, we can generate a defense exercise environment and train reinforcement learning (RL) based autonomous agents to attack the system described by the simulated environment. In this paper, we describe a two-player game-based RL environment that simultaneously improves the performance of both the attacker and defender agents. We further accelerate the convergence of the RL agents by guiding them with expert knowledge from Cybersecurity Knowledge Graphs on attack and mitigation steps. We have implemented and integrated our proposed approaches into the CyberBattleSim system.8 pagesen-USThis work was written as part of one of the author's official duties as an Employee of the United States Government and is therefore a work of the United States Government. In accordance with 17 U.S.C. 105, no copyright protection is available for such works under U.S. Law.Public Domain Mark 1.0UMBC Ebiquity Research GroupText