Volume IV - Issue II - April 2004
2nd Generation SPAM Filter Now Available
You Now Have an Effective SPAM Filter
Joseph West
Executive Vice President
Introduction

EZ-NetTools is excited to announce a new, second generation SPAM filter available for your EZ-NetTools email accounts. The new SPAM filter utilizes Bayesian statistics, the most advanced and customizable filter available today. It is reported to be 90 percent accurate at identifying SPAM. Our personal experience is even better.

Anyone who has an email account knows about SPAM. The latest reports in the media are suggesting as much as 70 percent of all email today is SPAM, and it is getting worse. The SPAMMERs have creatively side stepped most first generation, rule based or black/white list based filters. As quickly as a rule could be defined, they find a way to bypass it. Filters based on Bayesian statistics are much more insightful and adaptive to the changes.

Background

The term SPAM was first used as a label for junk emails in 1994. Two Arizona attorneys wrote a simple program to send mass emails to promote their business practice. This action angered many Internet users. One irate user posted a message saying, "Send coconuts and cans of Spam to Cantor & Co.(who had sent the junk email)". The term, SPAM, quickly became identified with junk email.

Since then, the amount of SPAM has grown enormously. Some of the first attempts to combat SPAM included generating a black list of spammers that email addresses and websites. Not only were the spammers able to quickly circumvent this approach but many innocent people were hurt with this "hand grenade" approach (anyone using the same or nearby hosting services were also black listed as well.) Another approach was to develop rules that would look for selected key words and sort the message out. Again, spammers quickly recognized and adapted by using variants to the keys words such as v*i*a*g*r*a. This method also resulted in many valid emails being flagged as SPAM, an event referred to as a "false positive." (In other words it was falsely identified as positive for SPAM.) A more sophisticated and adaptive method was needed.

Filters based on Bayesian statistics are very capable of distinguishing between valid emails and SPAM. Such filters adapt to new assaults easily. The underlying statistical principles were developed by a clergyman, Reverend Thomas Bayes, born in London in 1702. By analyzing the words, terms, URLs, etc found in a SPAM versus valid email, a mathematical probability that the word is SPAM is assigned. The filter is "taught" which words are likely to be found in SPAM messages and which are likely not to be SPAM. The filter then analyzes selected words in the message and makes a statistical decision about how to categorize the message. For example, if the term v*i*a*g*r*a is found in a message, it is guaranteed the message is SPAM. In fact, the more a spammer uses such strange variations of the word, the more certain the email is SPAM and the more easily the filter recognizes it.

The success of a Bayesian filter is based on how well you teach it. The filter is customized for each email account, a characteristic that makes the filter even more difficult for spammers to second guess. Our experience is that the filter is correct more than 90 percent of the time, with very few valid emails labeled as SPAM.

Features of EZ-NetTools Bayesian Filter

The EZ-NetTools SPAM filter has the following features.

  • The SPAM filter comes with a number of rules and black lists by default. It will identify many unwanted emails without you doing anything. Yet its real power comes from its ability to learn from you.
  • From the emails you have received you can checkmark those messages that you want the filter to learn are SPAM. In the same way you can mark messages that are valid (referred to as GOOD). You will need to mark both kinds to "teach" your filter. Even though the filter learns quickly, it needs be told about 250 good and 250 SPAM messages to get very good.
  • While you are viewing a message, there is a link to click to indicate the message as either SPAM or GOOD.
  • While you are viewing the message there will be a link to mark the sender as being either on a white list or black list. (When an email from someone on a black list is received, the filter will automatically treat it as SPAM regardless of the statistical scoring. The reverse is true of email from someone on a white list.)
  • You can specify if and how you want the subject line to be modified when the email is flagged as SPAM.
  • You can specify what folder to receive all messages marked as SPAM.
  • You can adjust the importance the filter places on selected rules if you find the rules are not functioning the way you want.
  • The EZ-NetTools SPAM filter will even filter your email if you normally download your emails to your PC using a program like Outlook or Outlook Express. The filter will occur before you request your emails. However, you will need to "train" the filter using the web based email system, EZ-OnlineMail. You will also need to occasionally empty your SPAM folder by using the web mail.
  • You can adjust how sensitive the filter is. If too many GOOD messages are marked as SPAM, you can make the filter less restrictive.
  • The more emails you mark as either SPAM or GOOD the more accurately the filter will work. While no filter is perfect, the EZ-OnlineMail filter will remove most of the clutter and allow you to focus on the real email. Remember, you will also need to clean out the folder where you store your SPAM messages.

    Setting up SPAM Filter on Your Account

    The SPAM filter is an add-on product for each email account. There will be a once a year charge for each email account. For example, you have three email accounts established for your domain name, you will need to purchase the SPAM filter for each email account. Contact your reseller for more information and pricing .

    Additional Information

    If the EZ-OnlineMail filter will not meet your needs for some reason, there are filters that can be added to your PC based email system. Some popular ones are Email Protect, SpamWasher, Spam Inspector, SpamAgent, and SpamBully. We have not analyzed or used these products, so, we provide the product names for informational purposes only and as a service to you. The exception is SpamBully, which worked very well in a limited environment.

    If you want to know more about Thomas Bayes or how Bayesian filters work visit the follow websites.

    Biography of Thomas Bayes http://www.bayesian.org/bayesian/bayes.html

    About Bayesian filters http://www.gfi.com/mes/wpbayesian.htm, http://www.process.com/precisemail/bayesian_filtering.htm