Classification and discovery of rule misconfigurations in intrusion detection and response devices

1. Classification and discovery of rulemisconfigurations in intrusion detection andresponse devices Natalia Stakhanova Yao Li Ali A. Ghorbani UNB Plato Consulting Inc. UNB

2. Problem • Signature-based Intrusion Detection Systems (IDS) are the most commonly used IDS in production environments • employ a set of attack signatures, i.e., rules • match incoming events to signature set • Quality of signature set is critical! • The potential problems: • Dependence on human expertise • Default signature sets • Adjustments over time

3. Problem • Is manual quality assurance possible? • Snort IDS v.2.8 > 9000 rules • Automatic support? • Research efforts for evaluating the firewall configuration, IPSec devices, routers [Fu01, Eronen01,Burns01,Gouda04,Hamed06,Al-Shaer05,Alfaro06]

4. Are IDS signatures different? • Absence of common format: • Cisco PIX Firewall rules: • outbound 11 deny 192.168.1.49 255.255.255.255 80 tcp • outbound 11 permit 10.1.1.11 255.255.255.255 0 0 • Snort rules: • alert tcp $EXTERNAL_NET any->$SQL_SERVERS 1433 (flow:to_server,established; content:"-|00|-|00|";sid:2000373;) • alert icmp $EXTERNAL_NET any -> $HOME_NET any (msg:"ICMP PING NMAP"; dsize:0; itype:8; sid:469; rev:4;)

5. Are IDS signatures different? • Use of logical operators: AND, NOT, OR • Firewall rules: AND • IDS: • Apply Default-Rule-Botnet: Potential Botnet Connection (DNS) on events which are detected by the systemand NOT when an event matches $DNS Serversand when the destination port is 53and when the attack context is Local to Remote

6. Are IDS signatures different? • Hierarchical representation • through rule options • e.g., “sid field” in OSSEC IDS, “flowbits” in Snort IDS • using direct reference to the other rules • e.g., QRadar • Example: • Rule 1: If protocol:tcp dstPort:995 flow:to server,established; content:”|16 03 00|”-> flowbits:set,sslv3.client hello.request • Rule 2: If protocol:tcp srcPort:995 flow:to client,established; flowbits:isset,sslv3.client hello.request; content:”|16 03 00|” -> Alert

7. Rule misconfigurations • Inter-rule misconfigurations • Conflicts between rules • Intra-rule misconfigurations • Conflicts within a rule, i.e., between rule fields

8. Inter-rule misconfigurations Non-hierarchical misconfigurations • occur between rules without direct hierarchical relationship • Redundant rules • the rules are redundant if all events that can match one rule also match another rule and both rules have the same response action • Verbose rules • the rules are verbose if they have the same action and their conditional part can be efficiently combined in one rule without affecting the classification of any event • Irrelevant rules • the rule is irrelevant if the event stream with the properties indicated in the rule cannot reach the IDS device. • Correlated rules • the rules are correlated if there is at least one event that matches both rules and both rules provide different response action • Exception rules • the rules are considered an exception if the rules have subset/superset relations and different response actions

9. Inter-rule misconfigurations Hierarchical misconfigurations • occur between rules without direct hierarchical relationship • Redundant dependency • refers to the situation when several interdependent rules rely on the output of a single rule. • Prerequisite steps? • e.g., multiple login attempts and port scanning • Circular dependency • occurs among rules that contain reference to each other in their conditional parts

10. Intra-rule misconfigurations • Redundant condition • repetitive condition within a rule Example: • Rule 1: If alertType: ACL Deny AND dstPort=1808-1810 log srcIP address • Rule 2: If Category is Access.ACL Deny AND destination is Database server AND dstPort is 1520-2430 create alert ‘‘ACL Deny’’ • Verbose condition • conditions that can be efficiently combined in one

11. Intra-rule misconfigurations • Complicated condition • condition that contributes to complicated and long structure of the rule. Example: Snort rule #9799, web-client.rule set: • Content field: 180 characters • Pcre field: 391 characters alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"WEB-CLIENT Panda ActiveScan PAVPZ.SOS.1 ActiveX clsid unicode access"; flow:established,to_client; content:"D|00|A|00|2|00|B|00|D|00|4|00|2|00|B|00|-|00|0|00|7|00|E|00|8|00|-|00|4|00|1|00|3|00|A|00|-|00|9|00|F|00|E|00|A|00|-|00|B|00|B|00|3|00|B|00|2|00|E|00|8|00|2|00|5|00|3|00|4|00|0|00|"; nocase; pcre:"/<\x00o\x00b\x00j\x00e\x00c\x00t\x00(\s\x00)*([^>]\x00)*c\x00l\x00a\x00s\x00s\x00i\x00d\x00(\s\x00)*=\x00(\s\x00)*(\x22\x00|\x27\x00|)c\x00l\x00s\x00i\x00d\x00(\s\x00)*\x3a\x00(\s\x00)*({\x00)?(\s\x00)*D\x00A\x002\x00B\x00D\x004\x002\x00B\x00-\x000\x007\x00E\x008\x00-\x004\x001\x003\x00A\x00-\x009\x00F\x00E\x00A\x00-\x00B\x00B\x003\x00B\x002\x00E\x008\x002\x005\x003\x004\x000\x00(}\x00)?\5/si"; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop; reference:bugtraq,21132; reference:cve,2006-5966; classtype:attempted-user; sid:9799; rev:3;)

12. Intra-rule misconfigurations • Needless condition • condition that only becomes true if the other conditions of the rule are satisfied • Contradicting condition • condition that contradicts the other rule conditions. Example: • Rule 1: If alertType: ACL Deny AND dstPort=1808-1810 log srcIP address • Rule 2: If Category is Access.ACL Deny AND destination is Database server AND NOT dstPort is 1808-1810 create alert ‘‘ACL Deny’’ • Irrelevant condition • condition that is irrelevant for event stream classification by this rule, • e.g., absence of service

13. Discovery of IDS Rule Misconfigurations • Intuitive approach: analyze rule relations • Model rule set as directed graph • each node denotes a specific rule of the set • the edges represent the relations of this rule with the other nodes

14. Abstract Rule Root No relations R10: Device: "Email" AND severity>5 AND confidence >5 create alert "Aggressive email sender" R9: dstIP: 172.234.15.5 AND dstPort:0-1024 log attempt R6: icmp AND TYPE: Redirect activate ICMPBlock in Firewall R13: alertType:"Aggressive attack attempts" OR magnitude>7 block srcIPaddress OR R8: tcp AND DstPort:80 AND GETcontent:"/something" blacklist srcIP address, set magnitude=9 R12: alertType:”Exploit attempt” OR ”Attack attempts” alert ”Aggressive attack attempts OR R11: alertType:"Suspicious event" OR "Port scan" alert "Attack attempts" OR R2: tcp AND RemoteToLocal AND DstPort:515 AND payload:"|24 7B 49 46 53 7D|" alert"Exploit attempt" R3: udp AND RemoteToLocal AND 65535 AND payloadSize>268 alert"Exploit attempt" R4: udp AND RemoteToLocal AND DstPort :515 AND payload:"|24 7B 49 46 53 7D|" alert"Exploit attempt" R7: tcp AND dstIP: 172.234.15.5 AND SrcPort=0 AND DstPort=6 alert"Suspicious event" R1: tcp AND RemoteToLocal AND stPort:139 AND payload:"|24 7B 33 5A|" alert"Suspicious event" R5: tcp AND srport:0 AND DstPort: 1024-3072 AND RSTFlag=1 alert"Port scan", set magnitude=8

15. Graph traversal • Idea: • DFS-based traversal • Combine rules in a bottom-up fashion • Analyze rules every time a new rule is added to the set

16. Experimental results • Evaluated rule sets: • open-source Snort IDS • Bleeding Edge Threats • set of rules provided by the vendor of the commercial IDS

17. Commercial IDS product • Number of rules: 45 • Number of detected misconfigutrations: 17 • Details of misconfigurations: • Exception: 6 • Correlation:1 • Redundancy:1 • Redundant condition:9

18. Snort IDS

19. Bleeding Edge Threats

20. Examples of misconfigurations

21. Processing time evaluation • 3 artificial rule sets: • full dependency • all rules with hierarchical structure interdependent • partial dependency • 50% of hierarchical rules • no dependency • no hierarchical rules 3.7 sec for 400 rules

22. Conclusion • Classification of the IDS rule misconfigurations • Algorithms for discovery of misconfigurations in IDS rule sets • Our results show that neither the commercial support of IDS nor the support of the security community allow to guarantee the absence of misconfigurations among IDS rules. • Although majority of the discovered misconfigurations are inefficiencies (warnings), rather than critical errors, they negatively affect IDS performance contributing to the complexity of the rule set and eventually leading to potential problems.

23. References • [Fu01] Z. Fu and S. F.Wu. Automatic Generation of IPSec/VPN Security Policies In an Intra-Domain Environment. In Proceedings of the International Workshop on Distributed System Operation & Management, pages 279–290, 2001. • [Eronen01] P. Eronen and J. Zitting. An expert system for analyzing firewall rules. In 6th Nordic Workshop on Secure IT Systems, pages 100–107, 2001. • [Burns01] J. Burns, A. Cheng, P. Gurung, S. Rajagopalan, P. Rao, D. Rosenbluth, and A. V. Surendran. Automatic management of network security policy. In Proceedings of the DARPA Information Survivability Conference and Exposition, pages 1012–1026, 2001. • [Gouda04] M. G. Gouda and X.-Y. A. Liu. Firewall design: Consistency, completeness, and compactness. In Proceedings of the International Conference on Distributed Computing Systems, pages 320–327, 2004. • [Hamed06] H. Hamed and E. Al-Shaer. Taxonomy of conflicts in network security policies. IEEE Communications Magazine, 44(3):134–141, March 2006. • [Al-Shaer05] E. Al-Shaer, H. Hamed, R. Boutaba, and M. Hasan. Conflict classification and analysis of distributed firewall policies. IEEE Journal on Selected Areas in Communications, 23(10):2069–2084, 2005. • [Alfaro06] J. G. Alfaro, F. Cuppens, and N. Cuppens-Boulahia. Analysis of policy anomalies on distributed network security setups. In Proceedings of the European Symposium on Research in Computer Security, pages 496–511. Springer, 2006.

24. Thank You! 24