Anti-Spam & Anti-Virus WiscMail Implementation University of Wisconsin - Madison

1. Anti-Spam & Anti-Virus WiscMail ImplementationUniversity of Wisconsin - Madison CSG Workshop September 21, 2004

2. Message Composition - Fall 2004

3. The Spam Threat • Users don’t want spam • Lost productivity • Offensive, Embarrassing • Legitimate messages get lost in the sea of spam • Spam isn’t going away • People buy from spammers • Legislation has not been effective • The SMTP protocol is inadequate • It allows spammers to forge message information • Spam is difficult to detect • Spammers learn how to get past filters • Legitimate messages WILL be lost

4. The Spam Threat • Anti-Spam is difficult to support • Users don’t like misclassifications • Client based anti-spam solutions interfere • Authorized mass-mailers want special treatment • Spammers use malware • Viruses “spam” themselves in mass quantities • Disinfected virus message clog Inboxes • Compromised computers DoS attack Anti-Spam services (RBLs) • Compromised computers send spam from inside the network

5. Anti-Spam Project Goals • Reduce spam by 80% from current levels • Users must be able to receive spam if they want (Opt-Out) • Provide an option to select levels of filtering • System must perform well and be scalable as message volumes increase • Provide a Web Interface to system • Compatible with existing infrastructure • Vendor supported system

6. Anti-Virus & Anti-Spam Integration • Why integrate anti-spam and anti-virus? • Faster processing • Messages are only opened once • Server consolidation • Virus messages can be treated as spam • Keeps the clutter out of the Inbox

7. How it works • Scan all incoming messages for spam and viruses • All potentially unsafe messages are scanned • Messages are marked with a spam “score” and then delivered as intended • Virus messages are deleted or disinfected • Filter the messages • Users choose whether or not to filter spam messages • Users choose what threshold (based on spam score) to filter spam

8. Spam Scanning • Allow mail from trusted sources to pass unaffected • All other mail is marked in the headers • e.g. X-Spam-Score: **** • 7 score levels • 0 asterisks means the message is likely not spam • 7 asterisks means the message is likely spam • Deliver all messages to recipient

9. Virus Scanning • Message is infected with a “junk” virus • e.g. netsky, bagel, mydoom, … • Delete messages without notification to sender or recipient • The induced message load from outbreaks causes delays for legitimate mail • Message is infected with a virus • Remove virus • Mark message as spam • Append [VIRUS] to subject • Message contains a suspicious attachment (exe, pif, scr, …) • Do nothing unless there is an outbreak • During an outbreak, treat these messages like viruses

10. Spam Filtering • Server-side filtering service • Custom built interface that allows users to configure individual filters to move messages into IMAP folders • Based on Sieve RFC • Compatible with IMAP and Web Mail users • ‘Junk Mail’ Folder • Reserved IMAP folder • Mail in ‘Junk Mail’ is deleted after 15 days of age

11. Spam Filtering • Junk Mail Filter • Users specify desired spam threshold (based on spam score) • Moves all spam marked at the specified level (or higher) into ‘Junk Mail’ folder • Accept List Filter • Keeps all mail from specified senders in the Inbox • Block List Filter • Moves all mail from specified senders to the ‘Junk Mail’ folder • Mailing Lists Filter • Keeps mail addressed to list addresses in the Inbox • Custom Filters • Users can create filters to move messages into IMAP folders • e.g. “If the Subject contains ‘CSG’ move the message into the CSG folder”

12. Spam Filtering - Issues • POP users see “disappearing email” • Mail is “POPed” from the Inbox only • POP users have to use Web Mail to see filtered mail • Alternatively, client-side filters can be used in conjunction with marked spam messages • Conflicts with client-based anti-spam filters • More misclassifications • Client filters are looking for spam that isn’t there • Support confusion • Users see two “junk” folders • Most new email clients have spam filtering enabled by default • Outlook 2003, Eudora 6, Mozilla variants

13. How Mail is Treated

14. Other Tools & Techniques • Server Filters • Similar to user-level filters, but applies to all messages • Saves load on spam and virus scanners, by deleting or rejecting at the front door • Hundreds of thousands of SoBig messages stopped during 2003 outbreak • Only works if the messages have definable characteristics • Site RBL • Real-time Blocking List • DNS Based • Allows us to dynamically block abusive computers from connecting to our mail servers

15. Other Tools & Techniques • Require SMTP Authentication • Compromised (zombie) machines are becoming the major source of spam • Rate Limit incoming and outgoing traffic • Limit abuse from spammers

16. Traffic Patterns

17. Traffic Patterns - Virus

18. Sophos PureMessage • Direct integration with our mail software (Sun iMS) • Also supports Sendmail and Postfix • Uses multiple spam detection technologies • Heuristics, RBLs, checksums • Customizable site policy based on Sieve RFC • Allows for specific actions based on message characteristics • Many message actions provided • Header/body modification, quarantining, discard, drop/replace attachments • Server cluster management

19. Sophos PureMessage • Honey potting • Dummy accounts set up to collect spam • Misclassification submittal process • Hourly automatic anti-spam heuristic updates • Hourly automatic anti-virus IDE file updates • Integration with anti-virus • End-user quarantine management • Not in use by UW-Madison

20. New Technologies • Sender Authentication by IP Addresses • What is it? • Helps prevent address spoofing • Allows administrators to specify the computers that are authorized to use addresses in a particular domain • Stores information in DNS • SPF • Open project (http://spf.pobox.com/) • 16% of mail domains have published SPF records • SenderID • Microsoft’s implementation, formerly called CallerID • proposal is having a hard time getting approved by the IETF • Relies on proprietary technology • The technology is still in too much flux to be fully embraced

21. New Technologies • Sender Authentication with Content Signing • What is it? • Helps prevent address spoofing • Uses SSL certificates to ensure that messages are sent by legitimate senders from the domain • DomainKeys • Specification submitted to IETF by Yahoo • Stores certificates in DNS • This technology is not as advanced as sender authentication with IP addresses

22. Anti-Spam Technical Alliance • Anti-Spam Technical Alliance • Yahoo!, Microsoft, EarthLink and AOL • Recommendations to Help Stop Spam • http://docs.yahoo.com/docs/pr/release1169.html • Address email address forgery with sender authentication • Recommendations for ISPs • e.g. rate limiting, limit port 25, close open relays, shut down zombie spammers • Recommendations for consumers • e.g. install firewalls and desktop a/v, make use of spam filtering technologies that are provided by their ISP

23. Future Plans • Sender Authentication (SPF) • Publish SPF records • Filter based on SPF • Possible use of quarantining • Advantages • Keep spam on spam servers instead of Junk Mail folder • Users can choose what to do with the messages that are quarantined • Users can correct the spam server so that it makes the right decisions in the future • Disadvantages • There are compatibility issues with our infrastructure • Users would have to learn yet another process

24. Question and Answer