Joshi, AnupamBhosale, Swapnil Mahendra2022-09-292022-09-292021-01-0112410http://hdl.handle.net/11603/25963This theses applies recent advances in NLP to anomaly detection in Windows OS. More specifically, we experiment using fastText as an embedding combined with an LSTM for state prediction. We explore whether we can model the normal process behavior on Windows and recognize deviations caused by malware. The actions performed by malware typically involve modifying the file system, modifying the Windows registry to change the system configuration & network actions. We developed a Windows kernel driver to capture file, registry, network events. We use fastText to train the embedding model to represent events as vectors. FastText learns not only the syntactic information but also semantic information hidden in the observed kernel events. The IP address, file path, process path, registry key etc. have syntactic structure and semantic relationships. Next, we train a sequence-based anomaly detection model using LSTM to learn the typical behavior of the Windows OS and the processes running in the system. Lastly, we propose a technique to identify measured windows event sequences as normal, or anomalies representing anattack. We evaluate the performance of this anomaly detection system to detect attacks on a system from their kernel level behavior. We collect datasets for normal (attack-free) and process takeover (attack) using the kernel driver system we develop, and use these to test our detection. We show that our approach has high accuracy, precision, and recall. We also propose to release our kernel driver to capture events as open source, to facilitate further research in this area.application:pdfThis item may be protected under Title 17 of the U.S. Copyright Law. It is made available by UMBC for non-commercial research and education. For permission to publish or reproduce, please see http://aok.lib.umbc.edu/specoll/repro.php or contact Special Collections at speccoll(at)umbc.eduanomaly detectionfastTextkernel driverminifilterwfpwindowsText