Anti-Spam

This document describes the spam-filtering program I wrote back in '97. That was the second version of the filter, completely rewritten from scratch. After '97, I've done quite a lot of smaller changes to make it function even better.

What I wanted when I started programming this, was a filter that should remove most of the Spam without me having to add addresses manually into a block file all the time. I did a lot of research on the Net, and didn't find any programs functioning as I wanted. I also wanted to do this filtering on my ISP's servers without me having to download all that crap.

Please note that this filter is made for the Qmail mail system, not Sendmail. It should be possible to modify it to work with Sendmail too, but I don't personally have neither the need or the time to do so. You're welcome to do that yourself if you want, and if you want to, I could upload that version too onto my homepage.

How the filter works

When you receive a mail, the qmail daemon will read a file named .qmail residing in your home directory. That file normally contains a couple of lines that can be used to forward your mail to another address, save the mail into a file on the server, or add it to the pop server for downloading with a mail client. It's also possible to run programs from the .qmail file.
The filter should be run as the first command (and first line) in the .qmail file. Depending on what it decides, it will tell the qmail daemon to continue reading the .qmail file or to just bounce the message back to the sender telling him that it didn't work.

The first thing the filter does, after being run by qmail, is to check whether your address appears in the To: or Cc: header fields. If it appears there, the mail is directly addressed, otherwise it's not, which means that it has either been sent to you by a mailinglist-server, or by someone not wanting you to know that it has been sent to other people as well. By default, directly addressed mail is let through, while other mail is blocked.
This behaviour can be controlled by using two files named .allow and .deny. Both of these are to reside in your home directory.

Worth noting is that mail sent using the Bcc: field is not considered directly addressed unless your address happens to appear somewhere else. Normally you won't see who it has been sent to. Not even that it has been sent to you. I consider mail that has been sent using Bcc: as spam, so if someone is sending me mail using that field he will have to re-send it. (Or you could add that person into your .allow file to make those mails pass through).

A short description of all files the filter uses

.allow

This file is used to allow non-directly addressed mail to pass through. This is most of the time used only for mailinglists. Here you can specify addresses or whole domains are to be let through. The addresses in this, and the .deny file are matched in the From:, To:, Cc: and envelope sender fields. If an address in this file matches, the mail will be let through.
Please note that this matching is done on everything in those lines, so it's a good idea not using too short strings in this file. For example, don't use just .com on one line, as it will match on everything containing that string, for example foo@mail.compaq.se. This shouldn't be a problem though, and I still see more pros than cons in using this matching technique.

.deny

Logically, this is the opposite to .allow. Here you can specify addresses that are to be blocked even if they are directly addressing you.
The notes above also applies on this one.
If a mail matches both the .allow and .deny fields, .allow is considered having higher priority. This can be useful when you want to block all addresses except one or a few on a domain. For example I'm using it to block everyone except abuse and two other addresses on hotmail.com. This can be done by adding hotmail.com into .deny and the addresses that are to be excluded from the blocking into .allow.

.isdirect

If you are forwarding mail from another address, you can add that address into this file, and the filter will use the addresses in this files when checking if a mail is directly addressed or not.

.killedmails

All mail that have been blocked are written into this file just in case something important got deleted. The format of this file is normal mbox format, so you can make a link from here to your mail directory and read it using pine.

.maillog

Logfile for the filter. One logfile for each address you have, ie. if you're using a .qmail-foobar file, mail to that address will be logged in the .maillog-foobar file.

.spam-bouncemessage

Here you can write an alternative message to be sent when bouncing a mail.

.spam-config

Configuration file for the filter. Here you can specify a few switches that changes the behaviour of the filter.

How to install this mess

First you'll have to compile the program. This is the easy part. Just run make and it should be done within a second or so.
Secondly, the program spamfilter to somewhere in your path, for example /usr/bin/ or in your homedirectory if you're on a system where you don't have write access in the /usr path on.
Now you probably should make a file named .spam-config in your homedirectory. This file can contain these options:
DEBUG=YES / NO - Used for debugging. Turns of deletion of mails, the filter will just log what it would have done.
BOUNCE=YES / NO - Bounce on / off. Set to no if you don't want any bouncemessages sent to people that got blocked.
SHORTLOGS=YES / NO - Long or short logs. There is also a DISABLE flag you can use, but only use it in an emergency, because blocked mail won't be saved.
The defaults are debug=no, bounce=yes and shortlogs=no.
shortlogs=yes can be nice to have when you have got everything working as it should. It turns off all unncessary logs. Only logs blocked mail.
Next thing to do is to create a .allow file. Start by adding the addresses to all mailing-lists you're subscribing to. Just take the address that appears in To: or From: depending on the listserver that's being used. One address per line. Do NOT add unnecessary spaces in this file, as everything except linefeed is included in the match. Comments are allowed by using a # as the first sign on a line.
If you are forwarding mail from other addresses to the filtered one, add those to the file .isdirect.
You can also make a .deny file at this point if you already have addresses that you want to block. This file has the same format as .allow and .isdirect.
All of these files I've described so far are optional. The filter will work without them, but probably not in the way you would like it to.
The only thing you need to do to get the filter working is to edit the file .qmail that, as all other files, resides in your homedirectory. This file should contain these lines:
|/usr/bin/spamfilter
That is, a pipe sign and the full path to the filter.
/qmail/user3/staham/Maildir/
That's the path to the Maildir used by the popserver.
Optionally you could use:
/home/staham/mail/Inbox
That one will write incoming mail to the mbox file Inbox.

Now the filter should work. Try sending a few mails to yourself to see that they go through. You can also check the logfile to see what it says.

Finally I just want to say that I won't take any responsibility if you happen to pipe your mail to /dev/null or something. You use it on your own risk. If your computer crashes, bursts into fire, or just dissappear into thin space, that's your problem, not mine. The filter has worked without a problem for me and a few other people for the last two years. I can't guarantee it will work for you, but it is unlikely that it won't. If it doesn't work, either try to fix it yourself by editing the source, or mail me and I'll have a look at it.

New versions of the spamfilter can be downloaded at http://www.algonet.se/~staham/linux/index.html.

Staffan Hämälä

Last modified: Sun May 9 13:02:09 CEST 1999