A Crawler-based Study of Spy-ware on the Web Lingfeng Mo

1. A Crawler-based Study of Spy-ware on the Web Lingfeng Mo

2. Spy-ware threaten Malicious spy-ware poses a significant threat to desktop security and integrity. But we talk about the threat from an Internet perspective. Using a crawler, they performed a large-scale study of the Web, sampling both executables and conventional Web pages for malicious objects.

3. Introduction Spy-ware has become the Internet’s most “popular” download. A recent scan performed by AOL/NCSA (America Online and the National Cyber Security Alliance ) of 329 customers’ computers found that 80% were infected with spy-ware programs. Results : 1: In a May 2005 crawl of 18 million URLs, Spy-ware in 13.4% of the 21,200 executables. 2. Scripted “drive-by download” attacks in 5.9% of the Web pages which were processed.

4. Where Spy-ware comes from? Spy-ware typically installs itself through one of two following methods: 1, download software to which piggy-backed spy-ware code has been attached->file-sharing software 2, visit a Web page that invisibly performs a “drive-by download” attack, the user’s browser will install software without the user’s consent.

5. Method of this study: • 1: Examine executable file content for piggybacked spyware programs and Web pages for drive-by download attacks. • 2: Analysis spyware • which areas of the Web are most infected • Which fraction of spyware that contains malicious functions (modem dialing, Trojan downloading). • 3: how spyware on the Web has changed over • time.

6. What this study did for Examine executable file ? • Determine whether a Web object contains executable software. How to determine? • 2. Download. install and rum executable files in VM. (Problems) • 3. Using commercial anti-spyware tool ?= executable contain piggy-backed spyware found by our Web crawler. How to do it? • Study spyware from: • 1: executable Web content contains spyware programs • 2: Web pages contain enbeded drive-by download attacks.

7. How to determine? 1. If Content-type HTTP header provided by the Web server when downloading the object was associated with an executable (e.g., application/octet-stream) 2. Extension with executables and installers (e.g., .exe, .cab, or .msi). 3. Also looked for well-known signatures at the beginning of the file to identify its type. If (exe.) {analyze}; else {not analyze}; Problem?

8. Problem in finding EXEs: • If EXE embedded in archives (ZIP,RAR)? • Solution: download->extracted EXE. From extension information • 2. EXE whose URLs are hidden in Java Script. • Solution: use web crawler scan and find URL. • If find any->add they to a list of pages to crawl. • GO TO SECOND STEP

9. Problems in installation: 1: Installation framework Solution: develop a tool that can Auto-click simple button (eg,. Next, install) . 2: Forms (E-mail, name, company) Solution: Auto-fill form with some dummy information GO TO THIRD STEP

10. Run anti-spyware tool (Lava Software AdAware) in a clean VM. • Collect infection analysis from its emitted logs. • Use log information to identify which spyware program were installed. • 4. Using online databases of spyware and also manually classified which function those spyware program contained. (Keystroker logging, Adware, Trojan backdoors, • or browser hijacking) • Limitation of this analysis method?

11. Limitations of this analysis method: • AdAware can only detect spyware programs that have signatures within its detection database, that is, our analysis misses spyware programs that AdAware does not find. • We only collected information about spyware software that is installed. Though many anti-spyware tools such as AdAware also identify malicious cookies or registry entries as spyware threats, we excluded these. • How to Analysis spyware?

12. Web Crawling Tool: Web Crawler (Heritrix: available at: http://crawler.archive.org/) 1. Crawled sites from 8 different categories in order to understand how spyware had penetrated different regions of the Web (adult entertainment sites, celebrity-oriented sites, games-oriented sites, kids’ sites, music sites and etc.) 2. Some random sites. How spyware on the Web has changed over time?

13. How spyware on the Web has changed over time? • Do the same above both in May and October. • Update the spyware database of Adaware.

14. Result: *While the absolute number of spyware-infected executables dropped substantially between the crawls, this is due primarily to a single site whose number of infected executables declined from 1,776 in May to 503 in October. *Overall, about 1 in 20 of the executable files we crawled ontained spyware, an indication of the extent of the spyware problem.

15. Spyware prevalence

16. Top 10 spyware programs and sites.

18. Are some Web categories more dangerous than others?

19. What kinds of spyware do we find?

20. Number of spyware programs installed

21. Defense: 1: signature-based tools 2: construct blacklistsof URLs or domains that are suspected to contain spyware

22. Can signature-based tools keep up? Re-analyzed all of the spyware-infected executables we found in October, but using the older AdAware signature database that was available in May. If new spyware threats were released between May and October, this older version of AdAware might not have signatures that match them. RESULTS SEE TABLE 5

23. Conclusion: it is important to keep an anti-spyware signature database up-to-date

24. How effective is blacklisting? overall conclusion we can draw is that blacklists are ineffective in two ways: many blacklisted sites contain no spyware, and many nonblacklisted sites do contain spyware.

25. Continue compare data: Conclusion: blacklisted spyware programs tend to contain a greater fraction of keyloggers, dialers, and Trojan downloaders. It appears as though blacklists tend to focus on spyware that contains more dangerous functions.

26. Summary 1: Our results show that spyware piggybacked on executables is a significant threat. 2: 1 in 20 of the executables we identified were infected with spyware in our October crawl – a surprisingly high fraction. 3: some Internet zones, such as game or celebrity sites, have a higher incidence of executable spyware than others. While some changes have occurred in the time between our crawls, at a high level they show a similar level of risk to Web users.

27. Limitations: • Did not crawl the entire Wen -> base on sampling. • 2. Can not explain any relationship between density and the • presence of threats on the desktop, since the latter is based • on the behavior of real users. • 3. Limited by anti-software (Adaware).

28. Comparing LBP to other local descriptors Why the tested methods work well with the easiest fb probe set? They are robust with respect to variations of facial expressions, whereas the results with the fc probe set show that other methods than LBP do not survive changes in illumination.

29. References: 1. Alexander Moshchuk, Tanya Bragin, Steven D. Gribble, and Henry M. Levy. A Crawler-based Study of Spyware on the Web, 2005 2. Ad-Aware. http://www.lavasoftusa.com/ software/adaware/. 3. Internet Archive. The Heritrix web crawler project. http: //crawler.archive.org/. 4. Webroot Software, Inc. Automated threat research. Described at http://research.spysweeper.com/automated_research.html.