FREE ELECTRONIC LIBRARY - Thesis, dissertations, books

Pages:     | 1 |   ...   | 2 | 3 || 5 |

«Abstract. Eight sites participated in the second DARPA off-line intrusion detection evaluation in 1999. A test bed generated live background traffic ...»

-- [ Page 4 ] --

The forensic information provided in identification list files was generally accurate for attacks that were correctly detected. Table 6 shows results for four highperformance systems that provided all optional identification information. The first column in this table shows the system type. The second column shows the total number of attacks detected by each system (at the highest false alarm rate) followed by a slash and the number of in-spec attacks that this system should have detected as specified in the system description. The first two expert systems both detected roughly 80 attacks each. They were combined systems that could have detected a maximum of roughly 170 in-spec attacks using both host-based and network-based input data. The third system used network sniffing data alone and thus had fewer inspec attacks (102) and the fourth system used only Solaris file-system information and thus had only 27 in-spec attacks. The remaining columns show the accuracy of the identification information provided for detected attacks. The third column shows the percentage of detected attacks where the attack category label was correct. The fourth column shows the percentage of detected attacks where the names of old attacks were correct. Participants were provided a list of names for old attacks before the evaluation was run which were used to label attacks. Items in this column apply only to old attacks that were detected. The next column shows the percentage of detected attacks where 90% or more of the victim ports were identified and the final column shows the percentage of detected attacks where all the source IP addresses were correctly identified.

–  –  –

This table shows that the additional identification information provided was generally accurate for attacks that were correctly detected. For example, for the first expert system, the attack category and name is correct roughly 90% of the time, and the victim ports and source IP addresses are correctly identified for more than 70% of the detected attacks. The upper three systems in Table 6 all used network sniffing data and provided good identification performance. The last Forensic analysis system, was a host-based system. Its good performance suggests that much of the identification information required can be obtained from a host-based analysis that doesn’ t rely on audit data.

All systems in Table 6 also provided attack start times as optional identification information. These times were computed by participating systems using off-line data with no constraints on look-ahead and thus they do not necessarily represent times that could be provided by real-time system implementations. Start time accuracy was generally good for R2L and DoS attacks. The attack start time latencies were less than 15 seconds for more than 80% of these attacks. Start time accuracy was not as good, and differed across systems for probe and U2R attacks. Start times were provided for probe attacks by the first three systems in Table 6. The third system (DMine) correctly identified the start of all probes to within 15 seconds while the first two expert systems had start time latencies that were often many minutes delayed for slower probes that spanned long time intervals.

The first two expert systems and the last system in Table 6 provided start times for U2R attacks. These attacks were unique because many of them included multiple separate telnet interactions separated by long time intervals and others were performed as part of long single telnet sessions containing many normal user commands. In attacks that included multiple telnet sessions, initial sessions were run at user privilege level to prepare for the attack. The actual attack, which provided rootlevel privilege on UNIX machines, was run only in following sessions. Results for the first two expert systems in Table 6 and for last Forensic analysis system differ dramatically for these U2R attacks. The first two systems detected the time instant where the attacker became root, while the Forensic analysis system traced the beginning of the attack either to the beginning of the first session where attack setup actions occurred or to the beginning of the telnet session where the attack occurred.

Start times for 6 of the 8 U2R attacks detected by the Forensic analysis system were within 15 seconds of true start times, while start times for more than 90% of the U2R attacks detected by the first two Expert systems were delayed by more than a minute from the true attack times. These results suggest that the Forensic analysis system is accurately correlating information across multiple network sessions to arrive at accurate start times while the two expert systems are using the time of the root-privilege elevation as a start time.

9 Discussion

The DARPA 1999 intrusion detection evaluation successfully evaluated 18 intrusion detection systems from 8 sites using more than 200 instances of 58 attack types embedded in three weeks of training data and two weeks of test data. Attacks were primarily launched against UNIX and Windows NT hosts. Best detection was provided by network-based systems for old probe and old denial of service attacks and by hostbased systems for Solaris user-to-root attacks launched either remotely or from the local console. A number of sites developed systems that detect known old attacks by searching for signatures in network sniffer data or Solaris BSM audit data using expert systems or rules. These systems detect old attacks well when they match known signatures, but miss many new UNIX attacks, Windows NT attacks, and stealthy attacks. Promising capabilities were provided by Solaris host-based systems which detected console-based and remote-stealthy U2R attacks, by anomaly detection systems which could detect some U2R and DoS attacks without requiring signatures, and by a host-based system that could detect Solaris U2R and R2L attacks without using audit information but by performing a forensic analysis of the Solaris file system.

Results of the 1999 evaluation should be interpreted within the context of the test bed, background traffic, attacks, and scoring procedures used. The evaluation used a reasonable, but not exhaustive, set of attacks with a limited set of actions performed as part of each attack. It also used a simple network topology, a non-restrictive security policy, a limited number of victim machines and intrusion detection systems, stationary and low-volume background traffic, lenient scoring, and extensive instrumentation to provide inputs to intrusion detection systems. One finding that should not be misinterpreted is that most systems had false alarm rates which were low and well below 10 false alarms per day. As noted above, these low rates may be caused by the use of relatively low volume background traffic with a time varying, but relatively fixed proportion of different traffic types. We currently plan to verify false alarm rates using live network traffic and a small number of high-performing systems. Live-traffic measurements will also be made to update traffic statistics and traffic generators used in the test bed. Results obtained with the DARPA research systems used in the evaluation also may not generalize to more recent research systems or to commercial systems. Performance with the 56 attack types used in the evaluation also may not be representative of performance with more recent attacks or with other attacks against different host machines, firewalls, routers, or parts of the network infrastructure. Further evaluations are required to explore performance with commercial and other research intrusion detection systems, with more complex network topologies, with a wider range of attacks, and with varying mixtures and amounts of background traffic.

Comprehensive evaluations of DARPA research systems have now been performed in 1998 and 1999. These evaluations take time and effort on the part of the evaluators and the participants. The have provided benchmark measurements that do not now need to be repeated again until system developers are able to implement many desired improvements. The current planned short-term focus in 2000 is to provide assistance to intrusion detection system developers to advance their systems and not to evaluate performance. System development can be expedited by providing descriptions and labeled examples of many new attacks, by developing threat and attack models, and by carefully evaluating COTS systems to determine where to focus research efforts.

A number of approaches to improve capabilities of existing systems are suggested by 1999 results. First, techniques should be developed to process Windows NT audit data to detect attacks by extending existing approaches from UNIX to Windows NT.

Second, host-based systems shouldn’ rely exclusively on C2-level audit data such as t Solaris BSM data or NT audit data. Instead they should also examine information in the file system and in commonly-used system logs. Systems that use file system information could be used on hosts such as Linux where there currently is no C2level auditing and on any critical host where auditing is not turned on for fear of performance degradation. Third, systems should analyze a wider range of protocols and TCP services. For some protocols, information contained in packet headers alone is insufficient, but the content of network transmissions must be extracted to determine the purpose of important network interactions. Fourth, approaches that can detect new attacks, including anomaly detection, should be extended to more hosts and network traffic types. Fifth, systems should provide more forensic information to analysts and extend the optional attack identification information provided by many systems in 1999. This forensic analysis could simplify the task of verifying each alert, determining attacker actions, and responding to an attack. It could also provide a valuable lasting record of attack-related events. Finally, other types of input features should be explored. These could be provided by new system auditing software, by firewall or router audit logs, by SNMP queries, by software wrappers, and by application-specific auditing.

Acknowledgements This work was sponsored by the Department of Defense Advanced Research Projects Agency under Air Force Contract F19628-95-C-0002. Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the United States Air Force.

We would like to thank Sami Saydjari for supporting this effort. Many involved participants made this evaluation possible including Dick Kemmerer, Giovanni Vigna, Mabri Tyson, Phil Porras, Anup Ghosh, R. C. Sekar, and NingNing Wu. We would also like to thank Terry Champion and Steve Durst from AFRL for many lively discussions and for providing Linux kernel modifications that make one host simulate many IP addresses. Finally, we would like to thank others who contributed including Marc Zissman, Rob Cunningham, Seth Webster, Kris Kendall, Raj Basu, Jesse Rabek, and Simson Garfinkel.


1. J. Allen, A. Christie, W. Fithen, J. McHugh, J. Pickel, E. Stoner, State of the Practice of Intrusion Detection Technologies, Carnegie Mellon University/Software Engineering Institute Technical Report CMU/SEI-99-TR-028, January 2000.

2. E. G. Amoroso, Intrusion Detection: An Introduction to Internet Surveillance, Correlation, Trace Back, Traps, and Response, Intrusion.Net Books, 1999.

3. K. Das, The Development of Stealthy Attacks to Evaluate Intrusion Detection Systems, S.M. Thesis, MIT Department of Electrical Engineering and Computer Science, June 2000.

4. H. Debar, M. Dacier, A. Wespi, and S. Lampart, An Experimental Workbench for Intrusion Detection Systems, Research Report RZ 2998 (#93044), IBM Research Division, Zurich Research Laboratory, 8803 Ruschlikon, Switzerland, March 9, 1999, http://www.zurich.ibm.com/Technology/Security/extern/gsal/docs/index.html.

5. Robert Durst, Terrence Champion, Brian Witten, Eric Miller and Luigi Spagnuolo, Testing and evaluating computer intrusion detection systems, Communications of the ACM, 42 (1999) 53-61.

6. C. Elkan, Results of the KDD'99 Classifier Learning Contest, Sponsored by the International Conference on Knowledge Discovery in Databases, September, 1999, http://wwwcse.ucsd.edu/users/elkan/clresults.html.

7. A.K. Ghosh and A. Schwartzbard, A Study in Using Neural Networks for Anomaly and Misuse Detection, in Proceedings of the USENIX Security Symposium, August 23-26, 1999, Washington, D.C, http://www.rstcorp.com/~anup.

8. S. Jajodia, D. Barbara, B. Speegle, and N. Wu, Audit Data Analysis and Mining (ADAM), project described in http://www.isse.gmu.edu/~dbarbara/adam.html, April, 2000.

9. K. Kendall, A Database of Computer Attacks for the Evaluation of Intrusion Detection Systems, S.M. Thesis, MIT Department of Electrical Engineering and Computer Science, June 1999.

10. J. Korba, Windows NT Attacks for the Evaluation of Intrusion Detection Systems, S.M.

Thesis, MIT Department of Electrical Engineering and Computer Science, June 2000.

11. Lawrence Berkeley National Laboratory Network Research Group provides tcpdump at http://www-nrg.ee.lbl.gov.

12. Richard P. Lippmann, David J. Fried, Isaac Graf, Joshua W. Haines, Kristopher R. Kendall, David McClung, Dan Weber, Seth E. Webster, Dan Wyschogrod, Robert K. Cunningham, and Marc A. Zissman, Evaluating Intrusion Detection Systems: the 1998 DARPA Off-Line Intrusion Detection Evaluation, in Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX), Vol. 2, IEEE Press, January 2000.

13. R. P. Lippmann and R. K. Cunningham, Guide to Creating Stealthy Attacks for the 1999 DARPA Off-Line Intrusion Detection Evaluation, MIT Lincoln Laboratory Project Report IDDE-1, June 1999.

14. MIT Lincoln Laboratory, A public web site http://www.ll.mit.edu/IST/ideval/index.html, contains limited information on the 1998 and 1999 evaluations. Follow instructions on this web site or send email to the authors (rpl or jhaines@sst.ll.mit.edu) to obtain access to a password protected site with more complete information on these evaluations and results.

Software scripts to execute attacks are not provided on these or other web sites.

15. P. Neumann and P. Porras, Experience with EMERALD to DATE, in Proceedings 1st USENIX Workshop on Intrusion Detection and Network Monitoring, Santa Clara, California, April 1999, 73-80, http://www.sdl.sri.com/emerald/index.html.

Pages:     | 1 |   ...   | 2 | 3 || 5 |

Similar works:

«Section 4 Environmental Consequences 4.9 Other Values 4.9.1 Wilderness For the purpose of defining whether the Proposed Action or any of the alternatives, including the No Action Alternative, would result in a potentially significant adverse effect, various criteria were considered such as the Federal Wilderness Act or potential impacts to State lands that are under the jurisdiction of the Division of Lands and Forests. Significance Criteria Potentially significant adverse effects may occur if...»

«Piezo Film Sensors Technical Manual Measurement Specialties, Inc. Sensor Products Division 950 Forge Avenue Norristown, PA 19403 Tel: 610.650.1500 FAX: 610.650.1509 Internet: www.msiusa.com e-mail: sensors@msiusa.com P/N 1005663-1 REV B 02 APR 99 TABLE OF CONTENTS – Introduction.................................................. 1 Background................................................... 1...»

«Information Guide Common canine poisons in the house and garden www.thekennelclub.org.uk Common canine poisons in your house and garden www.thekennelclub.org.uk What is a poison? A poison is a substance, which when introduced to an organism, is capable of producing an unwanted effect. When we talk about poisons many people automatically think of dangerous chemicals such as cyanide or strychnine, but forget about substances that are more commonplace, such as plant leaves that cause skin...»

«JOEL’S ARMY Jewel Grewe DIS CE R N M E NT MINIS T R IE S, INC. © Copyright 1991 by Discernment Ministries, Inc. Second edition, 2006. No part of this booklet may be reproduced in any form without permission in writing from the Discernment Ministries, Inc., except in the case of brief quotations embodied in critical reviews or articles. All Scripture quotations are taken from the King James Version of the Bible. Published 2006 Printed in the United States of America Individual copies...»

«RENAISSANCE REALISM RENAISSANCE REALISM JOSEPH MOSCONI A giraffe was given to Lorenzo di Medici by the Sultan of Egypt in 1487; sadly, it died soon after when it was led through a low doorway and hit its head. —Museum wall text, National Gallery, Washington, D.C. Jeepers! this glad piranha helpfully disbanded that tearful terrier. Yikes! an anonymous jaguar balefully stood beside that forlorn goldfish. Eh! that blatant insect haughtily taped this measurable caterpillar. Alas! that wise...»

«Mountain Storms by Max Brand Max Brand while his father dies in a fall within the mountains, twelve-year-old Tommy is left to fend for himself. via stamina, ingenuity, and a magnificent alliance with a grizzly cub whose mom he helped, the boy survives. whilst Tommy is older, he and the undergo enterprise down into the valley the place they hinder the killing of an unbreakable horse. The horse, the bear, and Tommy develop into legend and an day trip is fastened to catch or kill Tommy. means...»

«Every move you make tells a secret. This important book adds a new dimension to human understanding. Julius Fast teaches you how to penetrate the personal secrets of strangers, friends and lovers by interpreting their body movements, and how to make use of your powers. Why do you move the way you do? Does your body betray your thoughts? Can you enjoy love-making to its fullest? Are you a 'closed' or 'open' family? What are homosexual signals? What body language does a girl use to say 'I'm...»

«WPPSI-IV Interpretive Considerations for Charlie O. Jackson (11/27/2013) Interpretive considerations provide additional information to assist you, the examiner, in interpreting the child's performance. This section should not be provided to the parent or recipient of the report. Please review these interpretive considerations before reading the report. These interpretive considerations may suggest that you make changes to the report settings in Q-global. If you make changes to the report...»

«THE DOWNWARD SPIRAL OF STRESS: WHO PULLED THE PLUG? It is estimated that 75 to 90% of all visits to doctors are due to stress-related disorders. INADEQUATE DIET, INACTIVITY AND POOR MENTAL HABITS RESULT IN A DOWNWARD SPIRAL THAT MIGHT BE MORE ACCURATELY CHARACTERIZED AS A LONG, SLOW, DISTRESSING & AGONIZING DEATH WHAT MUST YOU DO? TAKE CHARGE! YOUR LIFE DEPENDS UPON IT. All living creatures are subject to and respond to stress. When they stop responding they are what we call “dead.”...»

«MA MAJOR RESEARCH PAPER CONSTRUCTING IDENTITIES THROUGH MUSIC VIDEOS: A CASE STUDY ANALYSIS OF TEGAN AND SARA'S MUSIC VIDEOS Jovana Jankovic Supervised by Dr. Jennifer Brayton The Major Research Paper is submitted in partial fulfillment of the requirements for the degree of Master of Arts Joint Graduate Program in Communication & Culture Ryerson University York University Toronto, Ontario, Canada May 10, 2013 AUTHOR'S DECLARATION I hereby declare that I am the sole author of this MRP. This is a...»

«FAIRFAX COUNTY KINSHIP QUICK GUIDE Contents Introduction Legal Assistance What documents am I likely to need for the child I’m raising? Custody Where can I file for custody or get more information? I cannot care for the child, but neither can his parents. What should I do? If I live in Falls Church city, where can I file for custody or get more information? Financial Assistance What types of financial assistance are available? How do I get child support for my child(ren)? How do I access...»

«Fun Appreciation Awards You do not have to apply for a formal award to show your appreciation of the Guiders you work with! These awards are meant to be mounted on heavy paper or poster board, and presented at a district event. Don’t forget to add the person’s name, date and their meaning. What is really special about these awards is that you can choose one that says something about the person you are giving it to. Is she new to the district? Give her a ‘Welcome Aboard’ award. Is she...»

<<  HOME   |    CONTACTS
2016 www.dis.xlibx.info - Thesis, dissertations, books

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.