Hitchhiker's World (Issue #8)
              http://www.infosecwriters.com/hhworld/

[=--------------------=] Alert Verification [=--------------------=]
[=------=] Determining the success of intrusion attempts [=-------=]

Authors: Christopher Kruegel (chris@cs.ucsb.edu) and 
	     William Robertson (wkr@cs.ucsb.edu)

Affiliation: Reliable Software Group
             University of California, Santa Barbara

--[ 1. Introduction

Recently, intrusion detection systems (IDSs) have been increasingly
brought to task for failing to meet the expectations that researchers
and vendors were raising. Promises that IDSs would be capable of
reliably identifying malicious activity never turned into
reality. While virus scanners and firewalls have visible benefits and
remain virtually unnoticed during normal operation, intrusion
detection systems are known for producing a large number of alerts
that are either not related to malicious activity (false positives) or
not representative of a successful attack (non-relevant positives).
Although tuning and proper configuration may eliminate the most
obvious spurious alerts, the problem of the vast imbalance between
actual and false or non-relevant alerts remains.

One problem is the fact that intrusion detection systems are often run
without any (or very limited) information of the network resources
that they protect. Marty Roesch, the developer of Snort [7,10],
routinely brings up this point in his whitepaper [6] and posts to
security mailing lists [8] and calls for an IDS that possesses
knowledge of the network components it defends. The classic example
that Marty uses is the scenario of a Code Red attack that targets a
Linux web server. It is a valid attack that is seen on the network,
however, the alert that an IDS raises is of no use because the service
is not vulnerable (as Code Red can only exploit vulnerabilities in
Microsoft's IIS web server). To mitigate this problem, Roesch
introduces a concept called RNA, real-time network awareness [6]. RNA
is based on passive network monitoring to establish an overview of the
hosts and services that are being protected. This overview contains
enough contextual information to distinguish between Linux and Windows
servers, thus enabling a "network-aware" IDS to discard a Code Red
attack against a Linux machine.

The problem is that the concept of network-awareness is not broad
enough to completely capture the complexity that is at the core of
excessive amounts of false alarms. When a sensor outputs an alert,
there are three possibilities:

1. The sensor has correctly identified a successful attack. This
alert is most likely relevant (i.e., a true positive).

2. The sensor has correctly identified an attack, but the attack
failed to meet its objectives (i.e., non-relevant positive).
 
3. The sensor incorrectly identified an event as an attack. The
alert represents incorrect information (i.e., a false positive).

Most people/sites are only interested in type-1 alerts. Although some
sites might be interested in failed attack attempts (type-2), the
corresponding alert should be differentiated from a successful
instance. The key idea of alert verification is to distinguish between
successful and failed intrusion attempts (both false and non-relevant
positives). While contextual information can be helpful to perform
this distinction (as we have seen in the example with the Code Red
worm above), it is not always sufficient. Consider a Code Red worm
attacking a patched Microsoft IIS server. In this case, it is not
enough to know which operating system the host is using, but it is
also required to know which application is running and which patches
have been applied.

Alert verification is a term that we use for all mechanisms that can
help to determine whether an attack was successful or not. This
information is passed to the intrusion detection system to help
differentiate between type-1 alerts on one hand and type-2 and type-3
alerts on the other hand. When the success of an attack is a priori
impossible (e.g., no vulnerable service is running) or cannot be
verified (e.g., the attack failed because incorrect offsets were
used), the IDS can react accordingly and suppress the alert or reduce
its priority.

The next section classifies different mechanisms to implement alert
verification. In Section 3, we present our implementation, which is
based on Nessus [5] and Snort [10]. With this configuration, we
demonstrate how Snort, an open-source network intrusion detection
system, was modified to utilize information provided by Nessus, a
popular vulnerability scanner, to significantly improve Snort's
detection accuracy. Section 4 gives more details on our experience
with the deployed tool. Section 5 concludes and outlines future work.



--[ 2. Alert Verification

Alert verification, that is, mechanisms to verify the success of
attacks, can be realized by modeling the expected "outcome" of these
attacks. The "outcome" describes the visible and checkable traces that
a certain attack leaves at a host or on the network (e.g., a temporary
file or an outgoing network connection). In the following, we describe
different ways to verify the success of attacks, classifying the
different techniques based on the infrastructure that is necessary to
gather and verify attack traces.

1. Passive: 

Verification mechanisms in this group depend on a priori gathered
information about the hosts, the network topology, and the installed
services. A description of the network installation is required and
can be, for example, specified in a formal model such as
M2D2 [4].

Given an alert, it is possible to verify whether the target of the
attack exists and whether a (potentially vulnerable) service is
running. For remote attacks, it is also possible to check whether
malicious packets can possibly reach the target, given the network
topology and the firewall rule configuration, or whether the target
host reassembles the packets as expected by the intruder (e.g., using
the tool by Shankar and Paxson [9]). Passive monitoring also allows
one to check for outbound network connections that are initiated from
the victim machine, possibly connecting back to the attacking
host. The real-time network awareness (RNA) approach [6] described
above would also fall into this class.

The advantage of passive mechanisms is the fact that they do not
interfere with the normal operation of the network. In addition, it is
not necessary to perform additional tests that delay the notification
of administrators or the start of active countermeasures. A
disadvantage of passive mechanisms are potential differences between
the state stored in the knowledge base and the actual security status
of the network. New services might have been installed or the firewall
rules might have been changed without updating the knowledge
base. This can lead to attacks that are tagged as non-relevant, even
though a vulnerable target exists. Another disadvantage is the
limitation of the type of information that can be gathered in
advance. When the signature of an attack is matched against a packet
sent to a vulnerable target, the attack could still fail for a number
of other reasons (e.g., incorrect offset for a buffer overflow
exploit). To increase the confidence in verification results, it is
often required to actively check audit data recorded at the victim
machine.

2. Active: 

Active alert verification mechanisms do not exclusively rely on
a priori gathered information. Instead, the verification process
actively checks for evidence of an attack.

In its basic form, mechanisms in this group require that a network
connection can be established to the victim machine. A network
connection permits scanning of the attack target and allows one to
assess whether a target service is still responding or whether it has
become unresponsive. It also enables the alert verification system
to check whether unknown ports accept connections, which could be
evidence that a backdoor is installed.

A powerful active verification mechanism is the use of vulnerability
scanners. When an attack has been detected, a scanner can be used to
check for the vulnerability that this attack attempts to exploit. We
have implemented an extension for Snort [7,10] that uses NASL scripts
written for the Nessus [5] vulnerability scanner. When a Snort rule
is triggered, the suspect packet and associated event data is queued
for verification. The appropriate NASL script is determined using a
match on the CVE IDs [1], and then the script is run against the
target machine. If Nessus detects that the target is not vulnerable to
the particular attack, the Snort alert is tagged as non-relevant. A
more detailed description of this process can be found in the next
section.

Active alert verification has the advantage, compared to passive
mechanisms, that the information is current. This allows one to assess
the status of the target host and the attacked service and to
recognize changes at the victim host that serve as an indication of an
attack. However, active actions are visible on the network and it is
possible that scanning has an adverse effect on one's own
machines. Port scanning consumes network bandwidth and resources at
the scanned host, and, in addition, tests run by a vulnerability
scanner might crash a service. The scope of these scans is also
limited; the identification of some evidence associated with an attack
might require local access to the victim machine. In addition, one has
to make sure that the alerts generated in response to the activity of
the vulnerability scanner are excluded from the correlation process.

3. Active with authenticated access:

Mechanisms in this group gather evidence about the result of an attack
using authenticated access to the victim host. The difference with
respect to the previous group of techniques is the fact that the alert
verification system presents authentication credentials to the
target host.

Active verification with authenticated access can be implemented by
creating dedicated user accounts with appropriate privilege settings
at the target machines. The alert verification system can then
remotely log in and execute scripts or system commands. This allows
one to monitor the integrity of system files (e.g., the password file
or system specific binaries) or check for well-known files that are
created by attacks (e.g., worms usually leave an executable copy of
the worm on the file system). In addition, programs to get interesting
forensic data such as open network connections (such as netstat), open
files (such as lsof) or running processes (such as ps) can be invoked.

The advantage of mechanisms in this group is the access to
high-quality data gathered directly from a target machine. The
downside is the need to configure this machine for authenticated
remote access. This might be cumbersome in large network installations
or when hosts with many different operating systems are used.

4. Active with dedicated sensor support

Mechanisms in this group require, in addition to authenticated access,
special auditing support installed at the target machines. This
auditing support can be operating system extensions or special purpose
tools, such as host-based intrusion detection systems. The differences
between using standard tools and relying on dedicated sensors is that
standard tools are common in most distributions. In addition,
dedicated sensors often need complex configuration.

Dedicated sensors can be used to monitor system calls issued by user
applications. This allows one to check for the spawning of suspicious
processes (e.g., shell invocations) or for the access to critical
files. As opposed to standard tools, which present a current snapshot
of the system, sensors can keep a record of malicious activity. This
enables the verification system to gather events that are only
visible for a short period of time, which could be missed by a
snapshot.
 
The advantage of dedicated sensor support is the ability to provide
the most detailed and accurate audit records. The drawback is the
effort required to install and configure these sensors, and the fact
that certain sensors are not available for all platforms.


One issue that affects all active verification mechanisms is the
problem that information is gathered directly from the victim
machine. It can be argued that an attacker can tamper with the
compromised system to eliminate suspicious traces or, at least, hide
her activity from the auditing system. There are different approaches
to this problem. One possibility is to run audit tools with privileges
that require administrative (i.e., root) access to be turned
off. This maintains the integrity of the sensor when the intruder
obtains user access only or manages to crash a service with a
denial-of-service attack. The sensors operate in a best-effort mode
and deliver accurate results as long as possible. Also, simply
disabling auditing is a suspicious action by itself. Another, more
secure option is the use of a more restrictive access control system
such as LIDS [2] or Security-Enhanced Linux [3]. These systems can
prevent the administrator from interfering with the audit facility and
physical access to the machine is required to change or disable
security settings.



--[ 3. Implementation

As mentioned above, we have developed an active alert verification
tool. It is implemented as a patch to Snort, which integrates the
Nessus vulnerability scanner into Snort's core to perform verification
of alerts. The modifications mainly consist of an addition to Snort's
alert processing pipeline which intercepts alerts to be passed to
enabled alert plug-ins. These alerts are queued for verification by a
pool of verification threads. This allows Snort to continue processing
events, while alert verification takes place in the background. 

For each alert selected for processing by a verification thread, the
CVE ID, a unique identifier for vulnerabilities which are assigned by
the Common Vulnerabilities and Exposures project [1], is extracted and
used as an index into Nessus' collection of NASL scripts. NASL is the
scripting language designed for the Nessus security scanner. Its aim
is to allow anyone to easily and quickly write a plug-in to test for a
certain security hole, which can then be used by the Nessus
scanner. If an appropriate NASL script is found, it is executed by an
embedded NASL interpreter against the victim host or network
identified by the current alert. The vulnerable status of the target
is extracted from the NASL interpreter's output and is used to flag
the detected attack as either successful or unsuccessful. The alert is
then queued for output by any enabled alert plug-ins. The result of
each verification is also cached for a configurable period in order to
reduce load on the network. When no appropriate NASL script is found,
the alert has to be flagged as undetermined.

Post-processing systems (e.g., alert correlation engines or system
administrator scripts) can then utilize this additional information
when performing their analysis upon the alert stream generated by
verification-enabled Snort sensors.

An overview of the current implementation is depicted in Figure 1.

  +-------------------+       +----------------------+       +----------------+
 +-------------------+|      +----------------------+|      +----------------+|
+-------------------+|+     +----------------------+|+     +----------------+|+
| Unverified alerts |+  ==> | Verification threads |+  ==> | Alert plug-ins |+
+-------------------+       +----------------------+       +----------------+
                                      ^ |
                                      | v
                           +-----------------------+
                           | Target host / network |
                           +-----------------------+

            Figure 1. Snort alert verification architecture.

Our implementation is available for download at
http://www.cs.ucsb.edu/~wkr/projects/ids_alert_verification/.


--[ 4. Evaluation

The current revision of our alert verification patch to Snort has been
evaluated on an experimental testbed with regards to its effectiveness
in reducing Snort's false alarm rate. Three machines were present on
this testbed:

1. an attacker machine
2. a target machine
3. a machine with an alert verification-enabled Snort sensor deployed

A variety of known vulnerabilities were introduced on the target
machine, and corresponding signatures to detect attacks using these
vulnerabilities were enabled on the sensor machine. A wide range of
attacks were run against the target by the attacker with alert
verification enabled and disabled to compare the number of false
positives produced by Snort. The results are shown in Table 1.


                        Alerts | True Pos. | False Pos.
                      +--------+-----------+-----------+
           Standalone |  6659  |     24    |   99.64%  |
                      +--------+-----------+-----------+
 Verification enabled |   24   |     24    |    0.00%  |
                      +--------+-----------+-----------+

            Table 1. Alert verification evaluation results.


As one can see, with Snort running in standalone mode, 6659 attacks
against the target machine were reported. However, because either no
vulnerable service was actually present on the target or the targeted
service was not vulnerable, most of these attack could not have been
successful and can thus be considered non-relevant. Only 24 of the
alerts produced by Snort were true positives, and we arrive at a false
positive rate (or, to be more precise, a non-relevant positive rate)
of 99.64%. With alert verification enabled, however, alerts which
attempted to exploit missing or invulnerable services were tagged as
such; thus, the false alarm rate for Snort with alert verification
enabled dropped to 0.00% and only the 24 actual attacks were reported.

To gather real-world attack traffic and assess the amount of alerts
that the system is capable of identifying as non-relevant in a more
realistic scenario, we deployed two honeypots. One of the honeypot
machines was running a standard RedHat 7.2 Linux installation, the
other one was running an unpatched version of Microsoft Windows 2000
Server. Both hosts had a considerable amount of services with known
vulnerabilities. The network link to both honeypots was monitored by
Snort-2.0.2, using its complete set of 2625 rules. 

During a period of 14 days, Snort reported 164,415 raw alerts
referring to attacks against the RedHat Linux machine and 79,198 raw
alerts referring to attacks against the Windows machine. Among these
raw alerts, we noticed a large amount of attacks related to the
Slammer and Nachia worms. Also, a large amount of scan activity
against ports commonly used by web proxy and socks proxy servers was
registered. We believe that these scans are performed by spammers that
use these proxies as mail relays. Given the raw alerts, the alert
verification process was capable of tagging 161,166 attacks against
the Linux host (98.3%) and 78,785 attacks against the Windows host
(99.4%) as unsuccessful. Although a default installation of Snort was
used, the numbers clearly indicate that real-world attack traffic
produces many false or non-relevant positives that can be suppressed
using alert verification.

The results shown above demonstrate that alert verification improves
the false positive rate of NIDS implementations. However, the current
alert verification implementation for Snort suffers from several
limitations. One is that the granularity of CVE IDs, which is somewhat
necessitated by the choice of Nessus as the verification component,
reduces the effectiveness of the tool as a whole. This stems from the
lack of other additional information, such as host architecture,
revision of the vulnerable program, etc. which could result in the
vulnerability testing script reporting the service as not vulnerable
when in fact it is. It is also worth noting that this limitation
generalizes to the fact that, barring implementation flaws, active
alert verification is only as good as the available verification
scripts, just as the quality of a signature-based IDS depends on the
quality of its signatures.

Another issue is that the classification scheme of vulnerable, not
vulnerable, or undetermined may, as members of the focus-ids mailing
list [8] have pointed out, not be expressive enough to capture
information that is relevant to network security officers.


--[ 5. Conclusions and future work

We propose alert verification as a process that is launched in
response to an alert raised by an intrusion detection system to check
whether the corresponding attack has succeeded or not. When the attack
has not succeeded, the alert can be suppressed or its priority
reduced. This provides an effective mean to lower the number of false
alarms that an administrator has to deal with.

We have developed an active verification system based on Snort and
Nessus. As the current implementation stands, it is a useful tool for
reducing the false alarm rate of Snort. There is, however, always room
for improvement, and in this spirit we have planned some future
directions for further development of our alert verification system.
One issue is the coarse granularity of CVE IDs, which we plan to
address by extending Nessus. Another planned area of development is
the possible integration of an a priori knowledge base along with
passive information gathering techniques to supplement the active
verification techniques.


--[ References

[01] CVE - Common Vulnerabilities and Exposures,
     http://www.cve.mitre.org/ 

[02] LIDS - Linux Intrusion Detection System, http://www.lids.org/

[03] P. Loscocco and S. Smalley, Integrating Flexible Support for
     Security Policies into the Linux Operating System, Freenix Track
     of Usenix Annual Technical Conference, 2001

[04] B. Morin, L. Me, H. Debar, and M. Ducasse, M2D2: A Formal Data
     Model for IDS Alert Correlation, Proceedings of the International
     Symposium on the Recent Advances in Intrusion Detection, 2002

[05] Nessus - Remote Security Scanner, http://www.nessus.org/

[06] RNA - Real-time Network Awareness,
     http://www.sourcefire.com/technology/whitepapers.html

[07] M. Roesch, Snort - Lightweight Intrusion Detection for Networks,
     Proceedings of the USENIX LISA Conference, 1999

[08] Security Focus Mailing List - focus-ids,
     http://www.securityfocus.com/archive/

[09] U. Shankar and V. Paxson, Active Mapping: Resisting NIDS Evasion
     Without Altering Traffic, Proceedings of the IEEE Symposium on
     Security and Privacy, 2003

[10] Snort - The Open Source Network Intrusion Detection System,
     http://www.snort.org