Machine learning in IDS

1. Machine learning in IDS March 15, 2004

2. Source Papers • T. Lane and C. E. Brodley An application of machine learning to anomaly detection, NIST-NCSC National Information Systems Security Conference, 1997 • J. Ryan, M. Lin, R. Miikkulainen Intrusion Detection with Neural Networks, MIT Press, 1998 • A. K. Ghosh, A. Schwatzbard and M. Shatz Learning Program Behavior Profiles for Intrusion Detection, USENIX Workshop on Intrusion Detection and Network Monitoring, 1999 • D. Endler Intrusion detection: Applying machine learning to solaris audit data, ACSAC'98

3. Two Major Approaches • Misuse detection – define intrusions ahead of time and watch for their occurrence • Can detect well-known attacks via patterns • Future attacks cannot be preemptively detected • Anomaly detection – detect behavior that deviates from normal system use • Learn a normal system activity profile • Can abstract information about normal behavior to detect attacks

4. Basic Terminology • Concept Drift – behavioral changes undergone by valid users during normal use • On-line systems • Run in real-time with users • Computationally expensive • Off-line systems • Run against stored user data at a scheduled time • Cannot respond in real-time

5. Paper #1 • IDS must learn characteristic sequences of actions • These sequences differ on a per-user basis • Characteristic differences between these sequences differentiate valid users from intruders • Use the sequence as the fundamental unit of comparison • Omit filenames for privacy and focus on behavior instead of content

6. Paper #1 • Parse the command stream into a token stream: > ls –laf > cd /tmp > gunzip –c foo.tar.gz | (cd \ ; tar xf -) becomes… ls –laf cd <1> gunzip –c <1> | ( cd <1> ; tar - <1> ) • This token stream is stored in the dictionary, along with a similarity measure and a set of system parameters

7. Paper #1 • Compute a numerical similarity measure for pairs of sequences that have close resemblance

8. Paper #1 • Collected data from four users • Experimented with different analysis methods • Sequence length had a major effect on accuracy • Dictionary must be kept small to avoid false positives, and for performance reasons • The problem of informed, malicious users • The system performed well, some caveats • No concept drift • Novice users

9. Paper #2 • Describes the NNID (Neural Network Intrusion Detector) • Works off-line, identifies behavior using the distribution of commands a user executes • Selected 100 commands to describe the user’s behavior

10. Paper #2 • A machine was selected that had 10 users, for a total of 89 user-days • The network was trained on 8 randomly chosen days of data and then tested against the remaining 4 days of data • Two separate tests were run • Identifying remaining vectors • Identifying randomly-generated vectors

11. Paper #2 • Identified user vectors 93% of the time • False alarm rate of 7% • Rejected 63% of the random user vectors • Had an anomaly detection rate of 96% • All the false alarms were the same user, and were attributed to lack of data

12. Paper #2 • Overall, the system was a success • How well does the system scale with more users? • To what extent does user behavior change over time?

13. Paper #3 • Three algorithms were experimented with: • Table lookup • Backpropagation network • Elman network • These three algorithms range from memorization to generalization

14. Paper #3 • Equality matching is simple but effective • Data is partitioned into fixed-size windows • For analysis, data is compared to a ROC (Receiver Operating Characteristics) curve • This curve is essentially an intrusive measure that calculates the probability of intrusion

15. Paper #3 • A backpropagation network attempts to learn from network behavior • Multiple networks were trained for each program, and the best was kept • Networks were fed random data to generalize everything as anomalous • Allows single anomalies, but recognizes sequences of anomalies

16. Paper #3 • An Elman network can recognize recurrent features in the input • Perform classification of short sequences of events as they occur within a larger stream of events • The Elman network was the least tuned, but most successful

17. Paper #3 • Overall results

18. Paper #4 • Utilized the Solaris SHIELD Basic Security Module (BSM) for user audit data • Perl script parsed the BSM data into separate audit files for four different users

19. Paper #4 • Testing data consisted of normal sessions, interspersed with simulated account break-ins • Number of signal features was reduced to 13 from 488 • Ideal window size was determined to be 6

20. Paper #4

21. Paper #4 • Ultimately, the best solution was a combination of both anomaly and misuse detection

22. Common Problems • If an intruder can breach the system during the learning phase, the system can learn the malicious behavior • All tests were performed against low user numbers • No real-world testing was performed

23. Summary • Creating system usage “fingerprints” is a valid methodology for IDS • Systems can be run both on-line and off-line depending on the configuration needed • Real-world testing required before implementation