VMM Based End Point Firewall

1. VMM Based End Point Firewall Raghunathan Srinivasan Advanced Computer Network Security Project Interim Report

2. Overview • Introduction • Need for secure computing • Related Work • Work done in VM based monitoring • Design • Current Status • Evaluation Criteria

3. Introduction • The Internet is a shared resource • Consists of millions of machines all over the world • Internet is now widely accepted and used for variety of applications • An Indirect consequence of this has been that PC have gained popularity

4. Uses of PC • The popular uses of PC’s are • Online Banking • Online Transactions • Communication • PC’s are used to authenticate a person • Shared Secret Problem • Can be stolen by malware

5. Software Vulnerability • A PC may contain miscellaneous, uncertified software • It is very difficult to discover and eliminate bugs in standardized and well documented software • It is very difficult to teach humans to create bug free code • Software will have bugs, and they will continue to get exploited

6. Exploits • Vulnerabilities in software layers are exploited by attackers to gain control of user machines • Hackers also use social engineering to trick users into installing malicious software • Prompting user to install a plug-in • Another trick is to send malware as part of e-mail attachments

7. Security Software • Anti-virus • Detects malicious code in the system • Not effective, can detect only known viruses • Firewall • Can be patched as it resides within the Operating System • Rootkits can bypass firewall and install their own network drivers

8. Disabling Firewall • W32/Bagz worm • Installs itself on a PC by means of social engineering trick • It proceeds to install its own network driver to bypass firewall • It then opens a backdoor to download and receive files

9. Disabling firewall … • Win32.Bagle.AU • Spreads through file sharing • Primarily through P2P networks • This worm can rename itself from one infection to another • It patches other programs to execute from another address space • It opens a backdoor on port 81 to download files from remote and receive commands

10. Disabling Firewall • Vulnerabilities in software allow attackers to provide inputs that cause error in software services that shut-down the firewall • sending unexpected data in the datagram packets • application that handles this data crashes • cascading effect on other Windows applications including firewall

11. VMM – Virtual Machine Monitor • Used for • installation management • simulation • software testing • Emergence of powerful desktops allows VMM to be incorporated into security solutions • VMM can offer security & Isolation • VMWare, XEN, LGuest, VirtualPC

12. VMM uses • Used to detect rootkits • Can be used to hide information • Private Keys • Credit card info • Can be used to restrict device access to a particular machine • A VM can be designated to use only particular devices and applications

13. Design Details • Attempt to implement a end point firewall along with its policies inside the VM layer • XEN and Ubuntu Linux will be used to implement the firewall • XEN is a type I VM • XEN is a very bulky software layer. It contains many modules for para-virtualization, OS scheduling, device management

14. Design • Implement a stripped down version of the VM that handles network operations • Conceptually similar to Microsoft’s VM • The advantage of this implementation is that the thin VMM layer containing only the firewall functionalities offers much better performance than a full fledged Hypervisor with multiple guests and a root partition

15. Design Applications OS Kernel VMM layer Firewall Detects anomalous network requests Hardware

16. Requirements • All network policies need to be installed inside the VMM. • These policies should not be configurable from within the OS running inside • Since network calls are passed through the VMM, it can monitor all traffic into and out of the operating system • Ensures against any malicious rootkit that opens up ports on the system

17. Requirements • a virus may patch on an existing application such as the browser • use it to open a port that the browser would not normally use • This attack is difficult to contain • this attack can be mitigated by specifying the ports that an application would normally use

18. Prevent Social Engineering? • A popular social engineering technique is to construct websites that look similar to popular banking sites • trick the customer into revealing his/her private secret • the user can be asked to enter the list of websites that are frequently visited • The user can also be asked to enter his/her interest category • A web search for these categories can be done to maintain a list of popular websites that deal with them

19. Preventing social engineering • If a user attempts to access any domain that is outside that of the specified interest area firewall denies connection • This also will block websites that open due to accidental clicking on advertisements

20. Current Status • Installed XEN on a machine • Reading documents as to how to modify the code

21. Evaluation Criteria • The system will be under attack from various rootkits • Can assume the presence of a rootkit detector • Preventing Buffer Overflows are difficult for the scope of this project • The ability of the system to detect any software that is opening an unauthorized port will be the most critical evaluation criteria • The firewall should be able to deny such a request • The firewall should also be able to detect if a program is attempting to bypass the installed network drivers • Performance of the system should not get impacted by more than 10 % in terms of • memory utilized, cpu overhead

22. Deliverables • Finish coding before start of November second week • Finish testing before November 2nd week ends • Project report Due on November 14th

23. Thank You