A Testing Model for Dynamic Malware Analysis Systems

1. A Testing Model forDynamic Malware Analysis Systems Hugues Normandin Frédéric Massicotte Mathieu Couture Frédéric Michaud Presentation to SECTEST2012 Affiliated with ICST 2012 April 21st 2012

2. What is a D-MAS ? • Detected.. infected.. We want to know the impact ! • Documents stolen ? • Passwords stolen ? • What is a D-MAS ? • Controlled environment in which dynamic malware analysis is performed to report on the sequence of actions (also called a sandbox) • The problem • How to test them if they are a test system themselves ?

3. Testing the unknown.. How ? ? ? ? ? ? • Test system themselves • We know nothing about the input. • We do not know what to expect. • How to choose the malwares for the test cases and how to validate the output? • We have a solution to partially address the problem. ? ? ? ? ? ? ? ?

4. Test Criteria (Malware Sample Selection) • Action Coverage • Possible actions taken by malware samples. • All Action Coverage (AAC) • Must be identified by more than one D-MAS. File Access Create file Delete file Delete itself Rename file Read file … Registry Create registry key Delete registry key Modify registry key … Network Protocols DNS HTTP HTTPS FTP IRC SMTP SMB … User Account Create user account Modify password Mutex and Hooks Use mutex Install hook Other Host Actions Modify firewall Steal password Use address book Use documents … Process Create process Stop process Inject code in process Services Create service Start service Stop service …

5. Test Criteria (Malware Sample Selection) • File Type Coverage • Mediums used to infect computer systems. • All File Type Coverage (AFTC) • Intersection of the file types supported by the D-MAS evaluated and our malware samples repository. JPG WMV ZIP HTM EXE SWF DOC COM MOV ICO PDF RTF Input = A set of malware samples that satisfies AAC and AFTC

6. Test oracle (Ouput) • How we validate the output ? with a test oracle • To verify whether or not the actions identified by the D-MAS match the ones specified in the oracle. • Several types of test oracles in the literature • But none fits our needs for testing a test system • A test oracle to evaluate the difference between reports. Output = Discrepancy Evaluation

7. X Y Z K W Q R J Design • 8 D-MASs • 46 selected actions • 2 test criteria (AAC & AFTC) • 74 malware samples Oracle Matrix of 8 x 46 x 74 Malware sample Discrepancy evaluation Normalization D-MASs

8. Discrepancy Summary At least 1 D-MAS disagrees Action is conducted, everyone agrees 0 or 1 D-MAS was able to provide this action Action is NOT conducted, everyone agrees

9. Discrepancies • Many many.. • 33.9% of the results (red cells) • Causes of discrepancies • Plain bugs • Pure crash • Information in the wrong report • Environment • Microsoft Outlook not configured • SMTP not well emulated for email submission • Post-analysis and filtering • Not filtered enough or filtered too much • Granularity of the information • Semantics • Creating a registry key vs adding an entry of type list • What does it mean to do something (side-effects vs direct actions)

10. Our future work • Test Criteria • Adding new ones (e.g. Packer) • Improve • Output • Oracle more precise • Action sequences • Try to use a traditional oracle • A True Oracle with our own malware 00:01:923 create_process (C:\malware.exe) 01:04:132 create_file (C:\svchost.exe) 02:31:143 create_process (C:\svchost.exe) • Windows API • Cmd.exe • On windows start • Etc.. How is it started? 00:01:923 create_process (C:\malware.exe) 01:04:132 create_file (C:\svchost.exe) 02:31:143 create_process (C:\svchost.exe) 00:01:923 create_process (C:\malware.exe) 01:04:132 create_file(C:\svchost.exe) 02:31:143create_process (C:\svchost.exe)

11. Traditional oracle (just a snapshot)

12. Questions ?