Near real-time threat assessment using intrusion detection system's data

Abstract

The concept of Intrusion Detection (ID) and the development of such systems have been a major concern for scientists since the late sixties. In recent computer networks, the use of different types of Intrusion Detection Systems (IDS) is considered essential and in most cases mandatory. Major improvements have been achieved over the years and a large number of different approaches have been developed and applied in the way these systems perform Intrusion Detection. The purpose of the research is to introduce a novel approach that will enable us to take advantage of the vast amounts of information generated by the large number of different IDSs, in order to identify suspicious traffic, malicious intentions and network attacks in an automated manner. In order to achieve this, the research focuses upon a system capable of identifying malicious activity in near real-time, that is capable of identifying attacks while they are progressing. The thesis addresses the near real-time threat assessment by researching into current state of the art solutions. Based on the literature review, current Intrusion Detection technologies lean towards event correlation systems using different types of detections techniques. Instead of using linear event signatures or rule sets, the thesis suggests a structured description of network attacks based on the abstracted form of the attacker’s activity. For that reason, the design focuses upon the description of network attacks using the development of footprints. Despite the level of knowledge, capabilities and resources of the attacker, the system compares occurring network events against predefined footprints in order to identify potential malicious activity. Furthermore, based on the implementation of the footprints, the research also focuses upon the design of the Threat Assessment Engine (TAE) which is capable of performing detection in near real-time by the use of the above described footprints. The outcome of the research proves that it is possible to have an automated process performing threat assessment despite the number of different ongoing attacks taking place simultaneously. The threat assessment process, taking into consideration the system’s architecture, is capable of acting as the human analyst would do when investigating such network activity. This automation speeds up the time-consuming process of manually analysing and comparing data logs deriving from heterogeneous sources, as it performs the task in near real-time. Effectively, by performing the this task in near real-time, the proposed system is capable of detecting complicated malicious activity which in other cases, as currently performed, it would be difficult, maybe impossible or results would be generated too late.