FREE ELECTRONIC LIBRARY - Thesis, dissertations, books

Pages:   || 2 | 3 | 4 | 5 |

«Abstract. Eight sites participated in the second DARPA off-line intrusion detection evaluation in 1999. A test bed generated live background traffic ...»

-- [ Page 1 ] --

The 1999 DARPA Off-Line Intrusion Detection


Richard Lippmann, Joshua W. Haines, David J. Fried, Jonathan Korba, Kumar Das

Lincoln Laboratory MIT, 244 Wood Street, Lexington, MA 02173-9108

Email: rpl@sst.ll.mit.edu or jhaines@sst.ll.mit.edu

Abstract. Eight sites participated in the second DARPA off-line intrusion detection evaluation in 1999. A test bed generated live background traffic similar

to that on a government site containing hundreds of users on thousands of

hosts. More than 200 instances of 58 attack types were launched against victim UNIX and Windows NT hosts in three weeks of training data and two weeks of test data. False alarm rates were low (less than 10 per day). Best detection was provided by network-based systems for old probe and old denialof-service (DoS) attacks and by host-based systems for Solaris user-to-root (U2R) attacks. Best overall performance would have been provided by a combined system that used both host- and network-based intrusion detection. Detection accuracy was poor for previously unseen new, stealthy, and Windows NT attacks. Ten of the 58 attack types were completely missed by all systems.

Systems missed attacks because protocols and TCP services were not analyzed at all or to the depth required, because signatures for old attacks did not generalize to new attacks, and because auditing was not available on all hosts.

Promising capabilities were demonstrated by host-based systems, by anomaly detection systems, and by a system that performs forensic analysis on file system data.

Keywords: intrusion detection, evaluate, attack, audit, test bed 1 Introduction The potential damage that can be inflicted by attacks launched over the internet keeps increasing due to a growing reliance on the internet and more extensive connectivity. Intrusion detection systems have become an essential component of computer security to detect attacks that occur despite the best preventative measures.

Comprehensive discussions of alternate approaches to intrusion detection are available in [1,2,16]. Some approaches detect attacks in real time and can be used to monitor and possibly stop an attack in progress. Others provide after-the-fact forensic information about attacks and can help repair damage, understand the attack mechanism, and reduce the possibility of future attacks of the same type. More advanced intrusion detection systems detect never-before-seen, new, attacks, while the more typical systems detect previously seen, known attacks.

Study IDs Attacks/ False Stealth Comments Victims Alarms Puketza 2 4/1 Yes/ No Automated Attacks and Simple 1994 Un- Telnet Traff

–  –  –

Table 1. Characteristics of past intrusion detection evaluations.

The widespread deployment and high cost of both commercial and governmentdeveloped intrusion detection systems has led to an interest in evaluating these systems. Technical evaluations that focus on algorithm performance are essential for ongoing research. They can contribute to rapid research progress by focusing efforts on difficult technical areas, they can produce common shared corpora or data bases which can be used to benchmark performance levels, and they make it easier for new researchers to enter a field and explore alternate approaches. System evaluations that focus on additional practical issues including cost, ease of use, and traffic handling capacity are also useful for determining capabilities of complete deployable systems.

Without careful evaluations, installing an intrusion detection system could be detrimental because it might lead to a relaxation of vigilance based on unproven assumptions concerning system performance. It might also lead to inefficient use of trained personnel if systems produce many difficult-to-analyze false alarms. A careful assessment of intrusion detection systems is essential to understand capabilities and limitations and construct an effective security posture that makes use of detection and prevention mechanisms.

It is difficult and costly to perform reliable, systematic evaluations of intrusion detection systems. As a result, few such evaluations have been performed. Table 1 summarizes characteristics of important past evaluations that have compared multiple intrusion detection systems. It includes early studies which describe a methodology that can be used for technical evaluations [4,18,19], the most recent and extensive system evaluation of commercial products that we are aware of [22], and the real-time [5] and off-line [12,14] components of the 1998 DARPA intrusion detection evaluation. The first column in Table 1 provides the first author and date of the study, the second column indicates the number of intrusion detection systems evaluated, and the third column provides the number of attack types used and also the number of unique victim machines attacked. The fourth column indicates whether the study analyzed the number of false alarms produced for normal background traffic and also the duration of background traffic used to measure false alarm rates. The next column indicates whether stealthy versions of attacks were used in an attempt to evade intrusion detection systems, and the final column provides additional comments on the study.

Results are not shown in Table 1 because many studies were informal and didn’ t provide detailed information and because metrics differ widely across studies. The primary performance metric in all studies is the attack detection rate for each attack type used. This metric depends on details of the attacks and on the specific version of the intrusion detection system that was tested. It also is insufficient when used alone.

It must be combined with false alarm rates for normal traffic to assess the human workload required to operate intrusion detection systems and dismiss false alarms.

False alarm rates above hundreds per day make a system excessively expensive to deploy, even with high detection accuracy. Unless a system provides forensic information which makes alerts or putative detections easy to analyze, security analysts will not trust alerts and may spend many hours each day dismissing false alarms. Low false alarm rates combined with high detection rates, however, mean that alerts can be trusted and that the human labor required to confirm detections is minimized. Only recent DARPA evaluations have measured false alarm rates with a large quantity of rich background traffic. Other important metrics used by some studies include cost of commercial systems, ease of software installation and use, traffic handling capacity, and run-time memory and CPU requirements.

As can be seen from Table 1, evaluations have become more complex and extensive over the years. Initial evaluations included few systems, few attack types, did not include stealthy attacks, and included little normal background traffic to evaluate false alarm rates. The 1998 off-line DARPA evaluation includes 10 systems, 38 attack types, weeks of rich background traffic, stealthy attacks, and also led to a corpus or data base of attacks and background traffic that is being widely used for evaluation and development of intrusion detection systems. The first two evaluations in Table 1 describe initial research programs designed to develop a methodology for intrusion detection evaluation [4,18,19]. Both studies incorporated scripting software to provide repeatability by automating generation of attacks and background traffic. Few attack types were used in these studies and background traffic consisted either of a small number of automated telnet or FTP sessions. Both studies demonstrated the importance of repeatability for intrusion detection system development. Initial low detection and high false alarm rates were improved by cyclical testing and development with repeatable attacks and background traffic. The second study [4] also noted that generating realistic normal background traffic was complex and time-consuming in heterogeneous computing environments.

Many product comparisons of commercial intrusion detection systems have been published in the past few years. The third entry in Table 1 is a recent comprehensive product evaluation. It includes three host-based and seven network-based commercial intrusion detection systems which were evaluated using more than 12 attack types and four victim machines. This study also included stealthy probe or scan attacks and stealthy packet modifications described in [17] designed to elude intrusion detection systems. This study did not provide detailed per-attack detection results, but mentions that no system detected all attacks and that stealthy attacks successfully eluded many systems. Most of the systems evaluated rely on attack “signatures” to detect old or known attacks. New signatures can often be added by hand or downloaded from a remote site. This evaluation focused on practical system characteristics such as ease of use and cost, and did not measure false alarm rates for normal background traffic. It did, however, use network load-generating software to demonstrate that some network-based intrusion detection systems fail to detect attacks at high network loads.

The last two rows in Table 1 are for real-time and off-line DARPA 1998 evaluations. As can be seen from the table, the off-line evaluation is the most complex performed to date. It was an initial attempt at a comprehensive evaluation which included background traffic to measure false alarm rates, many attacks, and more than eight different intrusion detection systems. This exploratory evaluation was limited. It included only intrusion detection systems developed under DARPA sponsorship, only attacks against UNIX hosts, and background traffic designed to be similar to traffic on one Air Force base. Six research groups participated in this statistically-blind evaluation to provide unbiased measurement of current performance levels. The off-line evaluation, performed by MIT Lincoln Laboratory, included weeks of training and test traffic, more than 300 instances of 38 attack types, and resulted in an archival 1998 intrusion detection corpus or database [12,14]. This corpus can be processed simultaneously at many sites to evaluate and develop research systems and it continues to be used for algorithm development and as a baseline for future evaluations. The real-time evaluation, performed by the Air Force Research Laboratory (AFRL), evaluated a smaller number of systems which have real-time implementations using a more complex network, fewer attacks, and four hours of traffic [5]. Results of the 1998 evaluation helped determine the strengths and weaknesses of alternative technical approaches and had a strong influence on DARPA intrusion detection research goals. Further off-line and real-time evaluations which build on the initial 1998 effort were performed in 1999. This paper reports on the results of the off-line 1999 evaluation. Results and lessons learned from the 1998 off-line evaluation are first summarized, the 1999 off-line evaluation is described, 1999 results are presented, and suggestions are provided for future evaluations. Further details on the 1999 off-line evaluation are available in [3,10,13,14].

2 Summary of the 1998 Off-Line Evaluation

The DARPA 1998 Intrusion Detection Evaluation was an initial attempt to perform a comprehensive technical evaluation of intrusion detection technology. As noted above, this evaluation had limited goals. It was designed to evaluate only DARPA funded intrusion detection technology, and not complete deployable intrusion detection systems or commercial systems. It was also designed to measure false alarm rates using background traffic similar to that on one Air Force base and to measure detection rates of remotely-initiated attacks against UNIX hosts. Figure 1 shows the current version of an isolated test bed network which was first developed for the 1998 off-line evaluation. Scripting techniques which extend the approaches used in [4,18] are used to generate live background traffic which is similar to traffic that

–  –  –

flows between the inside of one Air Force base and the outside internet. This approach was selected for the evaluation because hosts can be attacked without degrading operational Air Force systems and because corpora containing background traffic and attacks can be widely distributed without security or privacy concerns. A rich variety of background traffic is generated in the test bed which looks as if it were initiated by hundreds of users on thousands of hosts. The left side of Figure 1 represents the inside of the fictional Eyrie Air Force base created for the evaluations and the right side represents the outside internet. The 1998 evaluation did not include the Windows NT victim machine or the inside sniffer shown on the left of Figure 1, but instead focused exclusively on UNIX and router attacks. Automated attacks were launched against three inside UNIX victim machines (SunOS, Solaris, Linux) and the router from outside hosts. More than 300 instances of 38 different attacks were embedded in seven weeks of training data and two weeks of test data. Machines labeled “sniffer” in Figure 1 run a program named tcpdump [11] to capture all packets transmitted over the attached network segment.

Six research sites participated in the blind 1998 evaluation and results were analyzed to determine the attack detection rate as a function of the false alarm rate.

Performance was evaluated for old attacks included in the training data and new attacks which only occurred in the test data. Detection performance for the best systems was above 60% correct at and below a false alarm rate of 10 false alarms per day for both old and new probe attacks and attacks where a local user illegally becomes root (U2R). Detection rates were mixed for denial of service (DoS) attacks and remote-to-local (R2L) attacks where a remote user illegally accesses a local host.

Pages:   || 2 | 3 | 4 | 5 |

Similar works:

«Defining Defending Odysseus CHARLES ROWAN BEYE In March 2006, members of the lacrosse team at Duke University were accused of raping an erotic dancer whom they had hired for a late night party. As the event began to be played out in the nation’s media, the president of Duke, Richard Brodhead, was called upon to put a spin on the situation that would save the reputation of an institution that wanted very much to be a major league player in the academic world whilst having to be obedient to a...»

«J1800NH2/J1900NH2/J2900NH2 UEFI BIOS Manual UEFI BIOS Setup 1 Main Menu 2 Advanced Menu 3 Chipset Menu 4 Security Menu 5 Performance Menu 6 Boot Menu 7 Exit Menu i J1800NH2/J1900NH2/J2900NH2 UEFI BIOS Manual UEFI BIOS Setup Introduction The purpose of this manual is to describe the settings in the AMI UEFI BIOS Setup program on this motherboard. The Setup program allows users to modify the basic system configuration and save these settings to NVRAM. UEFI BIOS determines what a computer can do...»

«J. C. Ryle and Comprehensiveness PETER TooN MOST MODERN THEOLOGICAL statements concerning the existence of diversity or the fact of comprehensiveness in the Church of England or in the Anglican Communion insist that this diversity and comprehensiveness must be taken seriously on fundamental, theological grounds. It is regarded as totally unsatisfactory merely to trace the diversity to its sixteenth and seventeenth century roots and then plot its development. One is expected to confess that...»

«Worcester Writers Analyses of Denis Leary Robert Benchley Frank O’Hara Michael Riggieri Mathematical Sciences WPI Class of 2011 Advisor: James Dempsey Table of Contents  Denis Leary Analysis..1  Denis Leary Bibliography..8  Robert Benchley Analysis..9  Robert Benchley Bibliography..14  Frank O’Hara Analysis..15  Frank O’Hara Bibliography..23 Denis Leary is a self-proclaimed “asshole”, an Irish drunk, a Willem Dafoe look-a-like, a comedian, a highly revered television...»

«WWW.LOANOFFICERRECRUITING.COM ©2013 2014 EMAC Recruiting Academy WWW.LOANOFFICERRECRUITING.COM Welcome to our learning guide! Thank you for your interest the Loan Officer Recruiting Blueprint learning guide. Your curiosity will pay dividends! For many Sales Managers, recruiting is a real puzzle, especially in recruiting Loan Officers. Your primary expertise is managing your organization, pricing and closing loans on time, and delivering superiority in customer service. Until now, your...»

«Public Markets as a Vehicle for Social Integration and Upward Mobility Phase I Report: An Overview of Existing Programs and Assessment of Opportunities PREPARED FOR: The Ford Foundation SUBMITTED BY: Project for Public Spaces, Inc. Partners for Livable Communities 153 Waverly Place 4th Floor 1429 21st Street NW New York, New York 10014 Washington, DC 20036 September, 2003 212-620-5660 202-887-5990 2 Public Markets as a Vehicle for Social Integration and Upward Mobility Table of Contents 5...»

«google+ konto google+ konto Google+ Konto löschen | Yahoo Clever 31.01.2012· Seit wenigen Tagen habe ich ein Google+ Konto, das ich wieder löschen möchte. Bevor das Konto gelöscht wird, wird verlangt dass ich das Google-Konto einrichten Google+ Tutorial Bevor Sie Google+ nutzen können, müssen Sie zunächst ein Google-Konto einrichten, falls dies noch nicht geschehen ist. Hierzu sind nur wenige Schritte Google+ Konto löschen, Gmail jedoch behalten Google...»

«Trends in atmospheric concentrations and deposition of nitrogen and selected hazardous substances to the OSPAR maritime area Monitoring and Assessment Series OSPAR Commission, 2009: Trends in atmospheric concentrations and inputs OSPAR Convention Convention OSPAR The Convention for the Protection La Convention pour la protection du milieu of the Marine Environment of the North-East marin de l'Atlantique du Nord-Est, dite Atlantic (the “OSPAR Convention”) was Convention OSPAR, a été...»

«Contents Lobster 60 Well, how did we get here? Robin Ramsay Introduction That I do this was suggested by Dan Hind and he made a number of useful comments on this text, some additions to it and suggested the title. Essentially I took parts of my 1999 Prawn Cocktail Party, pruned them and topped and tailed them, with a new intro and conclusion. Here is an example of how being on-line changes things: were Lobster still a hard copy magazine I would never have thought it worth devoting ten pages of...»

«Intercom – Sociedade Brasileira de Estudos Interdisciplinares da Comunicação XVII Congresso de Ciências da Comunicação na Região Sul – Curitiba PR – 26 a 28/05/2016 O Empoderamento Da Mulher Através da Expansão Da Fotografia De Moda 1 Arantxa Von Appen2 Ana Amélia Brauner Perera 3 Universidade Católica de Pelotas, Pelotas, RS Resumo: Este trabalho busca citar e explicar as razões pelas quais a expansão da fotografia de moda em união com a publicidade e o movimento feminista...»

«Industry Offers a Carrot in Online Music Fight Page 1 of 6 June 8, 2003 Home Site Index Site Search/Archive Help Welcome, furdx Member Center Log Out Go to Another Section Quotes: Search: NYTimes.com Technology Industry Offers a Carrot in Online Music Fight By AMY HARMON ike a lot of music fans roaming the Internet these days, David Bishop registers one basic sentiment when he thinks about the record industry. They're a bunch of greedheads, he says. They've been really fat on what I think of as...»

«Publish detail Read Eventguide books document, also Download PDF Read Eventguide digital file READ EVENTGUIDE PDF Complete data published is really a hard copy manual thats printed READ EVENTGUIDE Document nicely bound, and functional. It operates as a reference manual skim the TOC or index, get the page, and stick to the directions detail by detail. The challenge using these sorts of documents is the fact that user manuals can often become jumbled and hard to understand. And in order to fix...»

<<  HOME   |    CONTACTS
2016 www.dis.xlibx.info - Thesis, dissertations, books

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.