How To Filter Unwanted Email from your RCI Account

By Hanz Makmur - Feb 26, 2003

 

Last modified: March 6, 2003

 

Introduction

Unwanted email a.k.a SPAM  is becoming a big problem today. They clutter one's Inbox and make it difficult to find your your legitimate email. Fortunately there are many ways to separate out the junk email from your Inbox. Some are centralized and others are designed for each individual email clients. The quality of these varies by the design. Some simply rejects junk emails, some separate them as they are received while other require registration before any email can be sent.

 

Here at the Department of Computer Science (DCS), Rutgers University, we use a centralized system based on a program called SpamAssassin. We have used it for every email addressed to user@cs.rutgers.edu successfully since early 2001. This program analyzes every incoming email and tags each email based on a rating system. The rating is added into the header of the incoming email for further user action. We feel that it is not an appropriate to preemptively reject or delete email classified as spam because the tagging system of Spam Assassin is not 100% accurate.  There is the a possibility for a false positive and we don't want to risk a loss of important email.

 

Many people who have an email account at cs.rutgers.edu also have accounts on RCI and want to have the same ability to tag junk email. With this in mind, the LCSR Computing Facility decided to provide its users and University community with a facility to tag junk email for RCI account.

 

What to do

 

The following steps will show you what to do to activate the spam tagging feature on your RCI account. These instructions are made for RCI users only. RCI uses maildrop for its filter and this needs a different syntax. See maildrop documentation if you need to learn more about this filter. If you need to know how to do this for another system, please look at: http://please.rutgers.edu/show/howto/.

 

To enable spam tagging on your rci account, you need to know how you would be reading your email. There are 3 popular ways to read email and they are via POP3, IMAP protocol or Web Interface. Below you will find 2 sections that will explain how to setup spam filter according to the way you read your email. The first is for POP3-based mail readers and the second is for IMAP-based and WEB-based mail readers. The POP3 instructions show you how to simply tag all incoming email when they are rated as spam. The IMAP and Web Interface instructions have extra added steps. All junk email are automatically moved into a folder called SPAM. If you prefer to set up your own local mail rules or filters, follow the POP-based instructions.

 

1. If you are using a POP3 based mail reader, 

 

1. Login to your rci account. If you see the Main Menu prompt, you need to get to shell by typing shell and pressing return. Otherwise, continue to step 2. This will take you a command prompt

 

 

2. At the command prompt, type the following:

 

This will copy a file called .mailfilter into your home directory and enable your mail to be checked for spam.

 

3. Run your POP3 based mail reader and prepare it for incoming email. All email tagged as spam will have additional header  X-Spam-Flag: YES added to it. You will need to setup a local rule or filter on your mail reader to catch for this new header. Basically you set the rule or filter to look for:

 

 

and tell your mail reader to move it to a folder or delete it or whatever you like. Because this filter is not 100% accurate, you are recommended to file your spam email  into a folder instead of deleting it for later inspection to make sure there are no false positives. To learn how to setup a filter and create a new folder, look at the online help that comes with your mail client or contact your support person. Below you will also find direct link to some documentation on how to create  mail rules and filters.

 

4. Test your spam filter
   
   To test the filter above, simply create a new mail containing this string:

   XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

   and send it to yourself. This mail will arrive as spam. When the email is classified as spam, you will not see the forwarded email arriving in your Inbox.
   
   

 

Note: To make using email a lot easier, you should consider switching to an IMAP-based mail client. An IMAP-based email clients allow you to access your email on multiple computers without worrying where the last time you read your email. No matter which computer you use, you will be able to see all of your email stored in the IMAP server. Please see New Brunswick Computing Services documentation at http://nbcs.rutgers.edu/newdocs/ for more info on how to configure your mail reader to use the IMAP protocol.

 

B. If you are using an IMAP or a WEB based mail reader

 

1.     Run your mail reader and create a subfolder inside your Inbox and call it SPAM. WARNING: Do not skip the folder creation part. This has to be done first!!! (Some mail reader requires that you Subscribe to the newly created folder to make it show up in your folder list.) Please look at the online help of your particular program on how to create a new folder if you do not know how to do this.

 

2.     Login to your rci account. On the command prompt, type the following. If you see the Main Menu prompt, you need to get to shell by typing shell and pressing return. Otherwise, continue to step 2. This will take you a command prompt

 

3. This will copy a file .mailfilter into your home directory and enable your mail to be checked for spam.

4. Test your filter. You can test your filter by sending yourself an email that you know is a junk email. These email are regularly sent to you without your consent. You will notice that all incoming email tagged as spam will automatically be stored inside your SPAM folder.

 

C. How your mail is checked for SPAM

 

For those of you who would like to know more about how your mail is checked, the process is simple. When your mail arrived on rci.rutgers.edu, the .mailfilter file is checked by the mail server and executed. Since we have indicated in the mail filter that we would like send incoming mail to spamfilter.rutgers.edu using spamc program, the mail is then sent to spamfilter machine where it is being processed.

 

Inside spamfilter machine, you mail is checked in a variety of ways. Using SpamAssassin rule base, the mail is checked against wide range of heuristic tests on mail headers and body text to identify spam. The following information is taken right from spamassassin.org website about what is done to the mail.

 

1. header analysis: spammers use a number of tricks to mask their identities, fool you into thinking they've sent a valid mail, or fool you into thinking you must have subscribed at some stage. SpamAssassin tries to spot these.

 

2. text analysis: again, spam mails often have a characteristic style (to put it politely), and some characteristic disclaimers  and CYA text. SpamAssassin can spot these, too.

 

3. blacklists: SpamAssassin supports many useful existing blacklists, such as mail-abuse.org,or others. Here at LCSR Computing, Rutgers University we have chosen to use black listing service of: sbl-xbl.spamhaus.org (zen.spamhaus.org).

 

4. Razor:Vipul's Razor is a collaborative spam-tracking database, which works by taking a signature of spam messages. Since spam typically operates by sending an identical message to hundreds of people, Razor short-circuits this by allowing the first person to receive a spam to add it to the database -- at which point everyone else will automatically block it.

 

Once identified as spam, your mail is tagged as such.  Header flags are added for end user post processing. In our case above, we are looking for X-Spam-Flag: YES in the header to identify the mail as spam. As a default setting, when a mail reach a score of 5, the X-Spam-Flag: YES is added to the header. If you prefer to adjust this default setting, you may adjust it by looking for header flag: X-Spam-Level: xxxxx. The number of x'es determine the score rating for each email. For example, you can look for X-Spam-Level: xxxxxxx which means you will consider an email a spam when it has a score of 7 or more.

 

D. Further Reading:

 

1.   Eudora Online Tutorial - http://www.eudora.com/techsupport/tutorials/

2.   Filtering Your Email with Outlook Express -  http://www.netassoc.net/techstuff/OE_5_filters.htm

3.     Creating Message Filters in Netscape 6.2x - http://www.helpdesk.umd.edu/topics/email/os/windows/messenger/4154/

4.     Mail Filtering with Outlook XP - http://www.helpdesk.umd.edu/topics/email/os/windows/outlook/4157/

5.     General How To Document: http://please.rutgers.edu/show/howto/