Abstract

One of the biggest problems in intrusion detection systems (IDSs) is the high rate of false positive and false negative. In this dissertation, we propose a framework with two novel approaches to reducing the alert error rate (AER,) which is a combination of false positives, false negatives, and repeated true alerts.

The first novel approach is based on the premise that in a complicated attack, intruders carry out a sequence of steps to violate system security policies, with earlier steps preparing for the later ones. The intruders' true actions are unknown to the IDS but can be inferred from the alerts generated by the IDS sensors. We demonstrate that as an extension of colored Petri-Net, the hidden colored Petri-Net (HCPN,) can describe the relationship between different steps carried out by intruders, model alerts and actions separately, and associate each system state with a probability (or confidence.) These features make HCPN especially suitable for discovering intruders' actions from partial observations---alerts---and predicting intruders' next goals.

The second novel approach fuses the output of our HCPN-based alert correlation component using the exponentially weighted Dempster-Shafer (D-S) theory of evidence. Our approach uses the D-S theory to combine beliefs about certain hypotheses under conditions of uncertainty and ignorance. It allows quantitative measurement of certainty in the detection results.

Evaluations using the DARPA IDS Evaluation dataset and the attack scenarios from the Grand Challenge Problem (GCP) show that our HCPN-based alert correlation approach has the potential to greatly reduce the total number of alerts and to reduce the false positive rates. Our alert fusion algorithm further improves alert quality over the individual HCPN correlators installed at the demilitarized zone (DMZ) and inside network sites.