Exploration of data mining and its practical application in decision making

Abstract

Central to the economic and political health of the world is the ability to make sense of massive amounts of data in a fast paced environment that requires decision makers to make split seconds decisions based on the information available to them at that time. What if the machine learning algorithms or data mining algorithms didn't provide the full story to these decision makers? The research presented in this dissertation looks at this problem from three different perspectives. Combining the concepts of data mining, information flow through social networks and cyber situational awareness, we discuss a pyramid that could aid in enabling effective decision making. At the base of the pyramid is data mining and how different feature sets contribute to the overall accuracy of the classification algorithms. Through the addition of key pieces of data, we are able to increase the accuracy of our classification algorithms to between 80% to 95%. One level higher on the pyramid, we explore what users do with information in their social networks during a regionalized event. We identify their communication patterns and their behavior when presented with new information. In the final study, a proposed framework suggests a set of operational data classes required for decision makers to make effective decisions. Harvesting the power of data mining in a comprehensive, expedient, and legitimate manner is important when that data is being utilized for cyber situational awareness. As we show throughout this dissertation, it is imperative that the right type of information is discovered via data mining and classification to accurately and rapidly enable cyber situational awareness.