1 / 32

Statistical Flow analysis

Statistical Flow analysis. Section 4.1 Network Forensics TRACKING HACKERS THROUGH CYBERSPACE. purpose. Identify compromised hosts Send out more traffic Use usual ports Communicate with known malicious systems Confirm / Disprove data leakage Volume of exported data Individual profiling

dalila
Télécharger la présentation

Statistical Flow analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistical Flow analysis Section 4.1 Network Forensics TRACKING HACKERS THROUGH CYBERSPACE

  2. purpose • Identify compromised hosts • Send out more traffic • Use usual ports • Communicate with known malicious systems • Confirm / Disprove data leakage • Volume of exported data • Individual profiling • Reveal • Normal working hours • Periods of inactivity • Sources of entertainment • Correlate activity exchanges

  3. Process overview • Defined • “Flow record—A subset of information about a flow. Typically, a flow record includes the source and destination IP address, source and destination port (where applicable), protocol, date, time, and the amount of data transmitted in each flow.” (Davidoff & Ham, 2012)

  4. Flow record processing system • Flow record processing systems include the following components: • Sensor—The device that is used to monitor the flows of traffic on any given segment and extract important bits of information to a flow record. • Collector—A server (or multiple servers) configured to listen on the network for flow record data and store it to a hard drive. • Aggregator—When multiple collectors are used, the data is typically aggregated on a central server for analysis. • Analysis—Once the flow record data has been exported and stored, it can be analyzed using a wide variety of commercial, open-source, and homegrown tools.1 1. Pg 161

  5. sensors • Sensor types • Network Equipment • Many switches support flow record creation and export • Cisco - NetFlow format • Sonicwall – IPFIX and NetFlow • Be cautious of “sampling” which is not comprehensive data • Standalone appliances • Used if existing network software does not support flow data • Software • Argus – Audit Record Generation and Utilization System • Softflowd • Yaf – Yet Another Flowmeter

  6. Sensor software • Argus • Two packages • Argus Server • Argus Client • Libpcap- based • Supports BPF filtering • Documentation specifically mentions forensic investigation • Argus’ compressed format over UDP • Softflowd • Passively monitor traffic • Exports record data in NetFlow format • Linux and OpenBSD • Libpcap- based • Yaf • Libpcap and live packet transfer • IPFIX format over SCTP, TCP or UDP • Supports BPF filters

  7. Sensor placement • Investigators often do not have much control over placement • Infrastructures should be set up with flow monitoring in mind but usually are not • Factors to consider • Duplication is inefficient and must be minimized • Time synchronization is crucial • Most flow records are collected on external devices such as firewalls but this ignores internal network traffic which can be valuable • Resources are important when planning, prioritize • Do not over load your network capacity

  8. Modifying the environment • Leverage existing equipment • Switches, routers, firewalls, NIDS / NIPS • Upgrade network equipment • If existing equipment will not work deploy replacements • Deploy additional sensors • Use port mirroring to send packets to standalone sensor • Network tap another option

  9. Flow record export protocols • Proprietary – Cisco’s NetFlow • Open source – IPFIX • Relatively new and not yet matured – better tools on the horizon

  10. netflow • Maintains a cache that tracks the state of all active flows observed • Completed flows marked as “expired” and exported as a “NetFlow Export” packet to a collector • Newer versions (NetFlow v9) are transport-layer independent: UDP, TCP and SCTP • Older versions only support UDP and IPv4

  11. ipfix • Extends NetFlow v9 • Handles bidirectional flow reporting • Reduces redundancy • Better interoperability • Extensible flow record data using data templates • Template defines data to be exported • Sensor uses template to construct flow data export packets

  12. sflow • Supported by many devices – not Cisco • Conduct statistical packet sampling • Does not support recording and processing every packet • Scales very well • Generally not very good for forensic analysis

  13. Collection and aggregation • Placement factors to consider • Congestion • Flow records generate network traffic and can intensify congestion • Choose location where this will cause low network impact • Security • Export flow records on separate VLAN if possible • Isolate physical cables • Encrypt using IPSec or TLS • Reliability • Consider using TCP or SCTP over UDP • Capacity • One sensor or many? • Analysis strategy • Can affect all of the above, plan accordingly

  14. Collection systems • Commercial options • Cisco NetFlow Collector • Manage Engine’s NetFlow Analyzer • WatchPointNetFlow Collector

  15. Collection systems continued • Open source options • SiLK – System for Internet Level Knowledge • Command-line • Most powerful – biggest learning curve • Collector specific tools – flowcap and rwflowpack • Flow-tools • Modular and easily extensible • Only accepts UDP input • Nfdump / NfSen • Collector daemon – nfcapd • UDP network socket or pcap files • Argus • Supports Argus format and NetFlow v 1-8 • NetFlow v9 and IPFIX not yet supported

  16. Analysis • Defined • “Statistics—“The science which has to do with the collection, classification, and analysis of facts of a numerical nature regarding any topic.” (The Collaborative International Dictionary of English v.0.48).” (Davidoff & Ham, 2012) • Purpose • Store a summary of information about the traffic flowing across the network • Forensic data carving does not apply • Still very useful

  17. Flow record techniques • Goals and resources • This should shape your analysis • Access available time, staff, equipment and tools • Starting indicators – triggering event • Example evidence: • IP address of compromised or malicious system • Time frame of suspect activity • Known ports of suspect activity • Specific flows which indicate abnormal or unexplained activity

  18. Flow record techniques continued • Analysis techniques • Filtering • Baselining • “Dirty Values” • Activity pattern matching

  19. Filtering • Important to narrow down a large pool of evidence • Remove extraneous data • Start by isolating activity relating to specific IP address/es • Filter for known patterns of behavior • Use small percentages of data for detailed analysis

  20. baselining • Advantage of flow record data vs full traffic capture • Dramatically smaller allowing for longer retention • Build a profile of “normal” network activity • Network baseline • General trends over a period of time • Host baseline • Historical baseline can identify anomalous behavior • Most flow patterns will change dramatically if host is compromised or under attack

  21. “Dirty Values” • Suspicious keywords • IP addresses • Ports • Protocols

  22. Activity pattern matching • Elements • IP address • Internal network or Internet-exposed network • Country of origin • Who are they registered too? • Ports • Assigned / well-known ports link to specific applications • Is system scanning or being scanned? • Protocols and Flags • Layer 3 and 4 are often tracked in flow record data • Connection attempts • Successful port scans • Data transfers • Directionality • Data coming in (something downloaded) or going out (something uploaded) • Volume of data transferred • Lots of small packets can indicate port scanning • Large amounts of data usually cause for concern

  23. Simple patterns • Many-to-one IP addresses • DOS attack • Syslog server • “Drop box” data repository on destination IP • Email server (at destination) • One-to-many IP addresses • Web server • Email server (at source) • SPAM bot • Warez server • Network port scanning • Many-to-many IP addresses • Peer-to-peer file sharing • Widespread port scanning • One-to-one IP addresses • Targeted attack • Routine Server communication

  24. Complex patterns • Fingerprinting • Matching complex flow record patterns to specific activities • Example: • TCP SYN port scan • One source IP address • One or more destination IP addresses • Destination port numbers increase incrementally • Volume of packets surpass a specified value within a given period of time • TCP protocol • Outbound protocol flags set to “SYN”

  25. Flow record analysis tools • flowtools • SiLK • Argus • FlowTraq • Nfdump / NfSen

  26. Silk • Rwfilter • Extracts flows of interest • Filters by time and category • Partitions them by protocol attributes • Generally as functional as BPF • Rwstats, rwcounts, rwcut, rwuniq • Basic manipulation utilities • Rwidsquery • Can be fed a Snort rule or alert file and it will figure out which flow matches it and writes an rwfilter to match it • Rwpmatch • Libpcap-based program that reads in SiLK-format flow metadata and an input source and save only the packets that match the metadata • Advanced SiLK • Includes a Python interpreter “PySiLK”

  27. Flow-tools • Variety • Flow export data collection • Storage • Processing • Sending tools • “flow-report” • ASCII text report based on stored flow data • “flow-nfilter” • Filter based on primitives specific to flow-tools • “flow-dscan” • Identifies suspicious traffic based on flow export data

  28. Argus client tools • Ra • Reads • Filters • Prints • Supports BPF filtering • Racluster • Exports based on user-specified criteria • Rasort • Sorts based on user-specified criteria • Ragrep • Regular expression and pattern matching • Rahisto • Generated frequency distribution table for user-selected metrics: flow duration, src and dst port numbers, byte transfer, packet counts, average duration, IP address, ports, etc

  29. Flow traq • Commercial tool by ProQueSys • Supports many formats and sniffs traffic directly • Users can • Filter • Search • Sort • Produce reports • Designed for forensics and incident response

  30. Nfdump • Part of the nfdump suite • Includes • Aggregate flow record fields by specific fields • Limit by time range • Generate statistics • IP addresses • Interfaces • Ports • Anonymize IP addresses • Customize output format • BPF-style filters Nfsen • Graphical, web-based interface for nfdump

  31. etherape • Libpcap-based graphical tool • Visually displays activity in real time • Colors designate traffic protocol • HTTP • SMB • ICMP • IMAPS • Does not take flow records as input

  32. Works Cited Davidoff, S., & Ham, J. (2012). Network Forensics Tracking Hackers Through Cyberspace. Boston: Prentice Hall.

More Related