Asia's Source for Enterprise Network Knowledge

Monday, May 27th, 2019


Enterprise spam filters 101

It's amazing how enterprise spam filters work: An email is checked for legitimacy many times before it arrives at its destination. The mail server checks that you're allowed to send it via its service and probably also checks whether it is part of a suspiciously large number of emails, which may indicate that you're spamming. The receiving mail server will check a variety of properties before it allows the message to enter the recipient's mailbox; is the sender's email or IP address on a blacklist and are the subject and content indicative of spam?

Mail filters block emails based on matching keywords and expressions as well as statistical analysis, such as the naive Bayes classifier, to calculate whether an email is spam. Bayesian spam filtering can even be trained on a per-user basis, learning what a user's typical emails contain. Image filtering is used to detect skin tones and specific body shapes (normally associated with pornographic images). Gmail also performs optical character recognition on mid- to large-sized images. When an email is downloaded by your email client, this too checks it to see if it matches any of the client's filters that are both automatically and manually configured. Yet despite all these filters, we all still receive varying amounts of spam.

Spammers use various techniques to avoid these filters. They reduce the effectiveness of Bayesian spam filtering by including large amounts of legitimate text to decrease the email's spam score, which is the number most spam analysis programs give each message based on its spam-like characteristics. Text can also be replaced by images or drawn using rows of Xs. Although blacklists are of some help, they require a lot of time to maintain, as spammers use hundreds of thousands of compromised machines to send their spam, meaning the list is always playing catch-up. Whitelists, on the other hand, tend to be too restrictive, often blocking genuine inquiries or emails from new contacts.

Despite numerous laws and regulations restricting or outlawing spam, it still remains economically viable for spammers as they have no real costs, illegally using the resources of ISPs for free. While spam remains profitable, the industry will continue to spawn ingenious methods for getting through the filters and checks aimed to stop them.

The role of catching the majority of malicious spam, that is spam associated with malware or fraud of some kind, has fallen to the major ISPs and email service providers, as they have the resources and traffic control to intervene. Spam throttling, whereby the bandwidth and resources assigned to processing possible spam is greatly curtailed, has a direct effect on a spammer's operations. Making it unprofitable is the only solution to reducing the scale of the spam problem. But, prevention can only go so far; one man's spam is another's important message.