So, it's been at least 10 years since the first spam appeared on the Internet.....

Yes, this topic has been beated to death so many times by now. But I thought I can used a summary on battling the lunchie meat.....

Email is transport on the Internet using a protocol called SMTP. Security is not a great concern in the design of SMTP. In fact, one used to able to telnet to the SMTP port (port 25) and send a email spoofing as someone else. Sendmail and etc fixed this problem now, but it demostrates the lack of authenication scheme in SMTP to verify the origination of a message.....

Than they open up the Internet for business, and spamming becomes an "industry" by itself. There just a "market" for it and cost is low. It invaded Usenet, violated the WWW, brutalized SMTP, and causing a lot of grief on the cyberspace.

In the early days, users were fighting back by complaining to the sys admin and most of the time it will be taking care of. This usually mean the offending party is warned and blocked off/suspended. But as spamming become overwhelming, most users get tried of it and stop complaining. It is futile to report 100+ spams on your mailbox everyday!

On the other hand, spammers become more and more evasive, they spoof with non-existing email address, use badly configured domain to relay their spam, and now even using virus/worm/trojan/p2p to create a distributed network of spam zombies to avoid policing from their ISP and lower cost.

As everyone is getting tried of the situation, many solutions are developed or proposed to battle it.
- Client-side mail filter that drops spam as they are downloaded from the ISP. Many difference algorithms are developed. (e.g. Bayesian, rules-based)
- Server-side email gateway software/managed service that inspect mails before deliver it to the recipient.
- White/Black list that only accept known good originators and reject bad guys.
- Regulation effort to outlaw spamming on the Internet.
- Challenge/response scheme, a challenge is sent to sender if a mail is received on a protected box. No mail delivery to user until correct response received.
- Fix SMTP with sender authenication (e.g. CallerID for email, SPF)
Taxing the sender in some form.

We know none of these technologies are perfect. The mail filters work in various degree of success. The problem is that spam sometimes still slip though or false positive happens when real messages got dropped. It often need to combine with other techniques to improve the effectiveness. Also, the filter works on the recipient side in a passive manner. It does not cut down on the wasted bandwidth on the Internet.

The white/black lists need quite a bit of management, and it often blocked innocent server/domain. Furthermore, the solution required intensive processing resource.

The legistration path only work in a regional basis. ie. It won't works on all the spam from Taiwan I got on Yahoo mail. Also, policy-maker often not addressing the issue.

The root of the problem is that spammers are sending mails spooling as someone else. And there is no infrastructue in the MTA to prevent it.

To me, the CallerID/SPF is a good direction if it got deployed widely on the Internet. It helps MTA to detect whether a message is sent from a valid user/domain, and block all senders with unknown identity. I believed a lot of spams these days are sent by domain spoofing, which this technology should take care a big part of them.

More links:
- Spam, Spam, Spam, Spam, Spam, The FTC and Spam
- The Economics of spam
- Comparing spam filters
- Filters that Fight Back
- Spam: The Phenomenon (A very good summary!)
- 82% spam on the net - more bad news