Semantic Interpretation of Structured Log Files
Loading...
Links to Files
Permanent Link
Author/Creator
Author/Creator ORCID
Date
2016-12-19
Type of Work
Department
Program
Citation of Original Publication
Piyush Nimbalkar, Varish Mulwad, Nikhil Puranik, Anupam Joshi, and Tim Finin, Semantic Interpretation of Structured Log Files, 2016 IEEE 17th International Conference on Information Reuse and Integration (IRI) , 10.1109/IRI.2016.81
Rights
This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
© 2016 IEEE
© 2016 IEEE
Abstract
Data from computer log files record traces of events involving user activity, applications, system software and network traffic. Logs are usually intended for diagnostic and debugging purposes, but their data can be extremely useful in system audits and forensic investigations. Logs created by intrusion detection systems, web servers, anti-virus and anti-malware systems, firewalls and network devices have information that can reconstruct the activities of malware or a malicious agent, help plan for remediation and prevent attacks by revealing probes or intrusions before damage has been done. While existing tools like Splunk can help analyze logs with known schemas, understanding log whose format is unfamiliar or associated with new device or custom application can be challenging. We describe a framework for analyzing logs and automatically generating a semantic description of their schema and content in RDF. The framework begins by normalizing the log into columns and rows using regular expression-based and dictionary-based classifiers. Leveraging our existing work on inferring the semantics of tables, we associate semantic types with columns and, when possible, map them to concepts in general knowledge-bases (e.g. DBpedia) and domain specific ones (e.g., Unified Cybersecurity Ontology). We link cell values to known type instances (e.g., an IP address) and suggest relationships between columns. Converting large and verbose log files into such semantic representations reveals their meaning and supports search, integration and reasoning over the data.