If you run a mail server with sendmail, and a user wants to forward his mail to another server, one way is /etc/mail/virtusertable:

joe@example.com         joe@gmail.com

As you probably know, the sender address of most spam mails belongs to someone - not the spammer of course. If you refuse to accept a spam email, you must never send back an error email to the alleged sender address. Otherwise, you have become a source of unsolicited email yourself.

When in our example, gmail.com does not accept an email, your server will send an error mail to the alleged sender address.

If you watch your postmaster mailbox, you will notice this every time the alleged sender address is invalid. You might be tempted to suppress these postmaster notifications with

define(`confDOUBLE_BOUNCE_ADDRESS', `')dnl

But this only supresses the double bounces. If the sender address is valid, someone will get an error email about spam he didn't send.

The solution is to not use virtusertable for external forwarding, but instead do it in /home/joe/.procmailrc:

:0
* ^From: Mail Delivery Subsystem <MAILER-DAEMON@example.com>
* ^To: <joe@example.com>
bounced-mails

:0
! joe@gmail.com 

In this case, the sender address is set to joe@example.com before the mail is forwarded to gmail.com, and any error messages go back to joe.

Despite its name, POPFile can sort mails using a couple of protocols, including IMAP. It monitors a configurable list of IMAP folders (usually your inbox), classifies the mails with a Bayesian algorithm, and moves them to one of the target IMAP folders. If the classification was wrong, just move the mail to the correct folder in your email client, and POPFile will learn from its mistakes.

POPFile has been sorting my mail into four folders named 00todo, 01important, 02later and 03trash for the last week. It now has an accuracy of 78%, and is especially good at getting rid of mailing list threads which I am not interested in.

Installation instructions for Linux, Mac OS X and MS Windows can be found on the QuickStart page. While the IMAP module is labelled as experimental, I did not have any problems with it so far. Contrary to the ImapInstructions, it comes with the standard installation of version 0.22.4. So skip the first part of the page, and start with the "How to get it running" section.

I often have to automatically trigger some server action from a remote machine, and I have used a password-less SSH key and a dedicated account on the server for this so far.

But it turns out that SSH can provide more security. In ~/.ssh/authorized_keys, you can specify options for every key. This not only allows you to turn off port forwarding, but also to specify a command which will replace the interactive shell, or the command passed to the SSH client.

This command can be a shell script which checks what the client wants to run and executes it if it looks OK, or it can start the rsync daemon.

So to automatically run rsync over SSH, create a password-less SSH key on the client, put the public key in ~/.ssh/authorized_keys on the server, and add the following at the start of the line. Do not include the line break, and put a space between this and the key string.

no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,
command="rsync --server --daemon --config=/etc/rsyncd.conf ."
ssh-dss keydata user@example.com

In /etc/rsyncd.conf, list the directories the client may access. Because rsync will not be run by root, chroot would not work.

use chroot = no 
 
[backup] 
path = /var/backup 
read only = yes 

On the client, you then run rsync like below to fetch the latest backup files. Note the double colon between the host name and the module name. If needed, you could append a directory or file path after the module name.

rsync -e "ssh -i $HOME/.ssh/id_dsa-backup" \
  -a --delete server.example.com::backup/ /data/server-backup
    

Of course, it's still a good idea to use a dedicated account on the server for this.

Since last weekend, the Web and mail servers for clasohm.com have been running on the new machine without too much problems. So let's look at the mail setup.

SpamAssassin 2.6 -> 3.0

After upgrading to the latest version of SpamAssassin, I got more spam than before! As I couldn't find any postings from disappointed users on the mailing list, and training the Bayesan filter did not help, I finally ran spamassassin -D <spam. Because SpamAssassin uses all installed tools and packages it can, but silently ignores those it cannot find, analysing the output is a bit tedious, but worth it. Looking for messages about skipped tests, I quickly found out what was missing. While I installed Mail::SpamAssassin, CPAN did not upgrade the DNS package. So SpamAssassin complained about an old DNS package in the debug output, and skipped all blackhole tests and other DNS-related checks. This was easily fixed by running cpan Net::DNS.

To further improve SpamAssassin's performance, install tools like Vipul's Razor or the Distributed Checksum Clearinghouses (DCC).

MailScanner and ClamAV

MailScanner sits on top of MTAs like Sendmail or Postfix, and scans for viruses and dangerous HTML before delivering mails. It supports a long list of virus scanners, including ClamAV. The way it works is pretty cool - you have one sendmail daemon listening for incoming mail and putting mails into the incoming queue, one sendmail daemon processing the outgoing mail queue, and MailScanner shovelling mails from the incoming to the outgoing queue. If a mail contains a virus, it is discarded or put into a quarantine directory. The great thing is that you don't have to modify your MTA configuration at all. The init script of MailScanner takes care of starting the two sendmail daemons in the right way. And the MailScanner configuration is very flexible, so you can have different settings for different domains hosted on the same machine.

Greylisting with DCC

With more than 1,000 spam mails a day, I decided to give greylisting a try. DCC both offers a Milter interface and a greylisting mode. To configure it, modify /var/dcc/dcc_conf like this:

GREY_ENABLE=on
GREY_SRVR_ID=32702
DCCM_ARGS=-G
DCCIFD_ENABLE=on

The last parameter is not for greylisting, but allows Spamassassin to use the dccifd daemon instead of spawning a new process for every mail. The server ID is copied from the "auto local greylist server-ID" in /var/dcc/ids.

To activate the greylisting check in sendmail, one translates /etc/mail/sendmail.mc with DCC's hackmc script instead of the usual m4 invocation. With RHEL, these lines in /etc/mail/Makefile do the trick:

%.cf: %.mc
        umask 022
        mv -f $@ $@.bak
        /var/dcc/libexec/hackmc -T $< > $@

The "-T" parameter tells hackmc to not greylist mail when SMTP AUTH is used, which would obviously be a bad idea.

To prevent loss of mail, it is important to extend the default whitelist. Currently, Evan Harris provides a good whitelist. DCC uses a different format, so here is the file I currently use. To activate it, add this line to /var/dcc/whiteclnt:

include white-puremagic

The rest is standard DCC setup - /etc/init.d/dcc and the daily execution of /var/dcc/libexec/cron-dccd.

After our server had a near-death experience last week (its hard disk disappeared after a reboot), a couple of friends and I have been busy setting up a new server during the last few days. Not only a good opportunity to switch from SuSE 8 to RHEL 4, but also for installing the latest and greatest software, and for setting things up in the right way.

ipac -> MRTG

To monitor network traffic, I have used ipac-ng so far (screenshot). It is good for monitoring traffic on a single host, but what I really wanted was the Multi Router Traffic Grapher. Last time, I just couldn't figure out how to use it for anything but monitoring external devices with an SNMP interface.

This time, I searched for "mrtg snmp howto" on Google, and came up with a PDF that described how to set it up without SNMP. You just need a script that returns one or two values, and configure MRTG to call it. Couldn't be easier. Here's part of the configuration and the script, and a screenshot.

mod_throttle -> mod_bwshare

Putting your photo collection into a Web server has the disadvantage that from time time a braindead mirroring tool comes along and hammers the server with a never-ending stream of requests. With Apache 1.3, I have used mod_throttle to block ill-mannered visitors. It keeps track of how many requests have been received from an IP address, and blocks or slows down any further requests once a limit has been reached. One problem with this module is that to reset his count to 0, a visitor must not make any request for a certain time.

Because mod_throttle has not been ported to Apache 2 yet, I had to look for a replacement. What I found is mod_bwshare, which uses statistical shaping to detect unusual request patterns. Instead of a timeout after which the counter is reset, it keeps track of the average number of requests per second, and allows clients to exceed the limit for some time (screenshot). Once they accumulate too much debt, further requests are blocked.

I first followed the installation instructions in the README. It worked and I even was able to make a shared module for the RPM-installed Apache, but as stated in the documentation, the shared module could not be configured from httpd.conf. Then I discovered apach2-mod_bwshare.rpm, which works quite well on RHEL 4.

Update: After switching the DNS records to the new server yesterday and getting some real traffic, Apache crashed two times. Probably mod_bwshare cannot handle graceful restarts - after I started doing full restarts, it seems to work fine.