Tuesday, December 23, 2008

Anti-Spam Technologies

Spammers and hackers are constantly shifting strategies and tactics to get around spam filters. As new tactics evolve, anti-spam vendors must layer their new technology on top of the old. The following are the four major types of anti-spam technologies:

Content filtering
Early solutions relied primarily on word lists, email signatures, and lexical analysis. For instance, “Viagra” is a word that’s often tagged by content filters. To adapt, spammers started spelling it with “1”s instead of “I”s, and added spaces. Later, they began to include HTML graphics instead of putting in text. Recently, spammers began to put their content in embedded PDFs; some email security vendors can filter the content of PDFs as well.

Behavioral analysis
This type of anti-spam technology used Bayesian analysis, statistical analysis and heuristics in order to predict spam. The onus for this type of technology often fell on administrators, who had to do extensive tuning and trial and error before getting satisfactory results. Bayesian filters also increased the likelihood of false positives.

Identity analysis
This looks at the identity of known spammers often referred to as “reputation analysis”. This is a promising technology, but may require email authentication to become more widespread. Also, zombie attacks can get around this type of defense.

Pattern detection
By analyzing patterns of traffic, as much as 80% of traffic can be thrown out as invalid. This reduces the load on email servers and downstream email filters. This type of detection also does not add to the rate of false positives. All these technologies can layer on top of one another to create an effective anti-spam filter. However, organizations need to implement a secure messaging solution that takes the encryption burden of the end users and intelligently does the right thing.

Spamming and Dark Traffic

Some anti-spam vendors rely on identity analysisor reputation analysis to block spam coming from certain IP addresses. To get around this, spammers have now implemented zombie attacks or botnets (robot networks), planting spyware or Trojans on unsuspecting machines, which then work as slaves to remote machines, which then carry out a spam campaign.

Instead of one machine sending out tens of thousands of emails from a single IP address, zombie attacks can have a thousand slave PCs and a thousand different IP addresses sending out ten to twenty emails each. Effective email security requires vendors to constantly add adaptive techniques to their anti-spam systems, assuring organizations that even new threats like zombie attacks won’t get through their systems and email security professionals don’t get those angry calls or emails about unwanted messages.

Today’s enterprises expect spam filters to catch at least 95% of spam; the best spam filters catch upwards of 99% of unwanted emails. Even more importantly, however, spam filters should rarely catch a valid, wanted email and throw it out. These “false positives” have caused many arguments between companies when an expected email has been filtered out and never delivered.

Experts recommend that the false positive rate be as close to 0% as possible, since false positives can cost organizations dearly. Spam is not the only threat to email servers. While spam is unwanted and often annoying, a bigger threat to networks today is malicious and invalid email traffic, referred to as Dark Traffic, which can actually damage an email system or a network.

Dark Traffic includes viruses, worms, and Trojan horses that are sometimes attached to otherwise valid-looking emails. The directory harvest attack (dHA) is often a precursor to spam, when a corporate email server is bombarded with thousands or even millions of random name combinations in order to determine valid email addresses.

Email denial of service (doS) attacks, malformed SMTP packets, and invalid recipient addresses are other types of dark Traffic. As mentioned above, spam and dark Traffic can have a huge effect on bandwidth, often representing 70-90% of all inbound email traffic. Stopping spam and dark Traffic is essential to scaling email infrastructure accurately and keeping network performance up.