Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation
Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement.Téléchargez la présentation en cliquant sur ce lien.Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.
E-mail Basics • 從 RFC 821/2821, E-mail 基本組成結構 • Message = an envelope + content • Content = mail headers + body • MIME extension • For the transmission of images, audio, or other sorts of structured data in electronic mail messages. • MIME document series [RFC2045, RFC2046, RFC2049]
典型 E-mail 系統運作圖 • Incoming SMTP Gateway Farm • Firewall Internet • Mail Filtering • BL/GL/WL • Auto-learn • Bouncing server • Mail Spool • server • SMTPauth • Outgoing SMTP Gateway Farm
垃圾郵件防治概論 • Q1: 何謂 SPAM ? • 為何會有那麼多 SPAM mail ? • Where did the SPAMmer get my e-mail address ? • Q2: 如何反制 Spam mail ? • Or, how does the E-mail system know if an incoming mail is a spam message ? • Q3: The big issues with filtering to date • “Spam mail 阻擋效率如何 ?” • “多少正常的 e-mail 也被阻擋?”
何謂垃圾郵件 (Spam Mail)? • Best description: "Unsolicited Bulk E-mail" • 簡單地說 –你不想要而且也沒有主動要求他人寄送給你 • In human terms: bulk e-mail you didn't want, and didn't ask for • Mailing lists, newsletters, "latest offers": not spam, if you asked for them in the first place • UCE/UBE • UCE = Unsolicited Commercial E-mail • UBE = Unsolicited Bulk E-mail
垃圾郵件防護概論 (個別認知差異)-- Truth Depends on Interpretation • MTA0 Filtering with H1(msg) Mail Spool Accept • MTA1 (or MUA1) Filtering With H2(msg) Discard • MTA = Mail Transfer Agent • MUA = Mail User Agent • MTA2(or MUA1)
Spam Mail 特性 • 透過密件傳送功能 (header BCC for bulk mailing)以規避大量傳送信件的限制 • Avoiding the header length limits • 使用經常改變且是假的送件者以逃避郵件過濾 • 使用假造的電子信件郵戳以逃避追蹤 (faked Received lines for preventing/confusing the spam source tracing) • Syntax Ok, Semantics Error • 有時候Spam mail 會夾帶病毒
Where did the SPAMmer get my e-mail address ? • UCE/UBE 散佈途徑 • 名單收錄 • www homepage, USENET news articles • account password files on individual servers • 其他不當途徑 ( program bug, 招募會員活動, …) • 找尋管理較鬆散的 mail relay • Domain Zone scanning ( DNS) • URL scanning (web pages ) • Dictionary attack
為何會有這麼多 Spam Mail ? • SPAM 來源 • The most common form of spam is commercial spam, where the user is hoping to make a profit. • Take as an example spam, where a user abuses the un-metered nature of email to send out millions of emails. • As the incremental costs of sending more emails to the spammer are almost zero, he can still make a profit even with a success rate of 0.0001 %.
Why Bother Filtering Spam?--Economical view • On a global scale use of bandwidth, CPU resources and people time are wasted. • Seems to be about 30% to 60% of mail traffic, and increasing • The spam recipients time is wasted as well, and receiving the spam may directly costing the user bandwidth fees from their ISP. • ISP's also dislike spam because it costs them time and money to deal with the complaints, and recover overloaded mail servers that sometimes crash under the load of intensive spamming.
Why Bother Filtering Spam? - Social view • Nearly impossible to unsubscribe • “unsubscribe” addresses work only 37% of the time, according to the US FTC • Opt-in vs. opt-out • Legal retaliation (求償) not possible in most parts of the world yet • Only possible in some regions (e.g., Korea, some states in US, etc.) • Taiwan (準備採用 Option-out) • 已經訂草案, 送交立法( 近年的報紙消息)
A Hybrid Model for Anti-spam and Related Issues Economy + Education + Law + Technology • Authentication • Confirmation • Identification • Verification Economy Anti-SPAM Hash-money Education Retaliation ((求償) • Privacy protection Law Technology • BL, GL, WL • Machine learning • Opt-in, opt-out • Cyber forensic
How Do The Spammers Feel? • Spam relies on low overheads and extremely cheap delivery • Disrupt the equation and they will give up! • Already hurting, according to CBS: • “[I’ve gone through] unbelievable hardships [to keep spamming] ... My operating costs have gone up 1,000% this year, just so I can figure out how to get around all these filters”
Anti-Spam Paradigm • View 1: Prevent your sites from becoming SPAM sources • Sender authentication • Vulnerability scanning • Access Control (e.g., Firewall, etc.) • View 2: Prevent SPAM from entering the mailboxes of your site • Tasks: identify/detect incoming SPAM messages • Sender authentication • Whitelist, Blacklist, Greylist, • Auto-learning (personalization) • Sub-views: server-side vs. client-side filtering
Measures to Avoid Becoming SPAM Victims-Other Suggested Administration Policies • Basic Measures • Prevention/Disable of open relay (SMTP) and/or open proxy • Open proxy : tcp port 80, tcp 3128, etc. • Enable anti-virus capability • Sender Authentication • Enable SMTPauth (or something of the like) • Sender Policy Framework (SPF)– • Enable advertised white-list from Sender (through DNS)
Subtype of Problem -Forged Sender • Good news: • SPF, SenderID, DomainKeys, etc. • Bad news • This might cost more money for authentication services in the future (e.g., anti-virus checking services) • It might not be effective to authenticate something like abc.123@yahoo.com or a13yotdfgfg@hotmail.com, etc. • i.e., there might still be too much overhead/loading for authenticating these possible faked sources (Cf. DNSSEC)
Introduction to Greylist • In 2003, Evan Harris proposed a novel idea, which he called Greylisting. • i.e., based on the observation that many unsolicited bulk mails are sent via open proxies and other mechanisms that do not involve proper mail transfer agents. • The main ideas of greylisting are as follows: • Mail from unfamiliar senders are temporarily rejected and should be retransmitted by their ISPs' SMTP clients. • Mails from familiar senders are passed immediately. • You canwhitelist friendly SMTP servers, and you shouldwhitelist your own network, otherwise your SMTP clients will have real trouble to send e-mail. • Whitelisting localhost is also a must.
Rationale behind using Greylist (cont) • Grey listing works by assuming that contrarily to legitimate MTA, spam engines will not retry sending their junk mail on a temporary error. • RFC 2821 says that the sending MTA should retransmit 30 minutes or later after a failure, but spam sent through an open proxy as well as some viruses and worms are not retransmitted. • A proper sending MTA will repeat a transmission after a temporary4yz rejection. • If spammers ever try to resend rejected messages, we can assume they will not stay idle between the two sends. • Odds are good that the spammer will send a mail to an honey pot address and get blacklisted in a distributed black list before the second attempt.
Sample Knowledge Representation - Greylist for E-mail filtering (cont.) # # Auto-whitelisted tuples #======================================== # Sender IP Sender e-mail Recipient e-mail Expire 210.58.229.92 <epaper@eslite.com.tw> <ysm@cc.nctu.edu.tw> 1100042183 AUTO # 2004-11-10 07:16:23 210.58.229.92 <epaper@eslite.com.tw> <yytzou@cc.nctu.edu.tw> 1100042354 AUTO # 2004-11-10 07:19:14 210.58.229.92 <epaper@eslite.com.tw> <wsfu@cc.nctu.edu.tw> 1100041397 AUTO # 2004-11-10 07:03:17 • … more
Considerations, Caveats, and Differences • False positives (不是 SPAM, 卻沒有放行) are possible if a legitimate SMTP client does not retransmit an embargoed (禁制) message after the embargo expires. • Failing to retransmit clearly violates the SMTP standard, because it will result in lost mail should something other than Greylisting cause a temporary failure response. • False negatives (是SPAM, 卻未擋下) are common. • Greylisting can only detect bogus SMTP clients. Bulk mail advertisers that use standards compliant software automatically pass
Machine Learning Approach for Anti-Spam(cf. , SpamAssassin Concepts) • Combines many systems for a "broad-spectrum" approach • Lots of rules to determine if a mail is spam or not • Detect forged headers • Spam-tool signatures in headers • Text keyword scanner in the message body • DNS blacklists • Razor, DCC (Distributed Checksum Clearinghouse), Pyzor • These are combined to produce an overall score for each message • If over a user-defined threshold, the mail is judged as spam • Spammers cannot aim to defeat 1 system; the others will catch them out
SpamAssassin GA Start Good Enough? Yes Final Scores No Evolve Scores
In the diagram below, MTA refers to Mail Transfer Agent, or your mail server software: Postfix, Sendmail, Exim, etc. LDA refers to the Local Delivery Agent: Procmail, Maildrop, etc.. BEFORE: [MTA] ---> [LDA] ---> (User's Mailbox) AFTER: [MTA] ---> [DSPAM] ---> [LDA] ---> (User's Mailbox) \ \--> [Quarantine] [End User] ------> [Web UI] Dspam As a Delivery Agent Proxy
