Memory Scanning

1. Dr. Richard Ford rford@fit.edu Memory Scanning

2. What are we going to talk about? • Szor 12 • Virus Scanners – why they need to scan memory and what issues there are in this area

3. Virus Scanners • Look for “known” viruses • But they also must scan memory for viruses… why? • Why isn’t just using an “on-access” scanner good enough?

4. Advantages for Virus • Speed of spread (more infection opportunity) • Longevity of infection: active stealth • Imagine an integrity checker or virus scanner operating with a virus in memory…

5. DOS Scanning • DOS uses the Intel Processor in Real Mode! • Which means what? • DOS does not protect or differentiate memory from any process • No support for “virtual” memory

6. DOS Solution… • Can scan memory byte by byte • Effective • Not too slow (not much memory) • Direct access by the virus scanner (no OS involvement) • Problem is false positive rate – so scans using MCBs to reduce memory coverage • Safe?

7. But Life is Harder… • Later versions of MS OSes changed things around a lot • Virtual memory (basically, unlimited memory) makes life much harder • To understand why, need to understand how Windows NT and above manages memory

8. Why Virtual Memory? • Not a requirement for an OS • But… • Allows an application to not worry about usage (i.e. makes overlays uneccessary) • Provides process isolation • Provides memory protection • No memory limitation • Really needs dedicated CPU support – and Intel provides it

9. NT Pages • In NT, memory is broken down into pages, each 4096 bytes in size • The operating system handles how these pages are stored and translates into physical memory • This means that two processes can access the “same” address but two totally different physical pieces of memory • Introduces the idea of a “page fault”

10. Scanning memory in User-mode • If Process A is isolated from Process B how can you scan memory globally? • One solution: ReadProcessMemory() • API used by Debuggers, requires a “Handle” to a particular process • How to list processes… non-trivial! • Problem: requires us to basically provide global access to the user doing the scanning – that’s foolish! • Better to use kernel-mode… but this is hard too!

11. How Viruses use Memory • A virus can hook itself into the OS in a few different ways • User-mode and kernel-mode • Infect a DLL that’s used everywhere (like KERNEL32.DLL)

12. Memory scanning and paging • In order to scan memory in user-mode, the OS will have to generate a page fault for swapped memory pages • Slow! • Tempting to enumerate process and scan the disk images – why is this not a good idea?

13. Memory Disinfection • How? • Yeah, really… it’s not easy at all • Terminate the process that is infected • Detect and terminate remote threads • Patch “in memory” to deactivate (useful when the virus is loaded into every process) • Think carefully how memory protection on shared DLLs works

14. Deactivating a Filter • Let’s think about a virus which is a File System Filter Driver • Can’t unhook safely • How to disinfect?

15. Read-Only Kernel Memory • Can now mark memory as “Read Only” • How to disinfect? • Can use assembly within the kernel – but requires us to be kernel-mode

16. Problems for the Future • Encryption! • In-memory polymorphic code • Metamorphic in-memory • Code which “moves about” in memory (ouch) • Inject into multiple processes at once • In-memory stealth techniques

17. Generic Detection • Many ways to detect malware “generically” • Checksummers • Behavioral analysis • However, both have issues with false positives • Can create a more “optimistic” set of rules • Both these detection techniques appeared VERY early in the game

18. Heuristics • Not confined to static analysis, but most common there • Typically, we define a heuristic as a rule of thumb • Heuristic analysis was also used pretty early on in the detection game • If you think about it, a heuristic to detect our polymorphic hello world programs isn’t a bad idea…

19. Next… • Botnets and other loose ends