Christian.net Thursday
July 24 2008

Help Documents: PEP

Main | Actions | Headers | Tests | Commands | Reply Files | DNS Blocklists | SpamAssassin | Challenges | Glossary | Spam FAQ | SMTP Tutorial | PEP Wizard

Challenges

This page provides more detailed information about using PEP's "challenge" feature. It is assumed that you know the basics about creating a mailrule file and constructing rules.

The Basic Premise

A great deal (most?) of the spam today either has an invalid return address or it's a throw-away account that has already been abandoned by the spammer. If you could delete messages that come from such addresses, you'd be a long way towards eliminating spam from your mailbox.

The challenge feature "freezes" messages that come from unrecognised senders and only "thaws" and delivers them to you if the sender responds to a challenge message. If the challenge goes unanswered, the frozen message is eventually deleted (after 7 days by default) and you never see it.

How It Works

This description may be a little technical for some folks. It is not necessary to understand how it works in order to use it. You can skip this section if you prefer.

When a message arrives, PEP examines the Envelope-To: header to see if it contains a special challenge ID. If it does, then the following events occur:

  • PEP looks for a file in the "pep-challenge/indexes" directory of your account that has a filename the same as the challenge ID. This file will contain one or more unique message ID codes, one for each frozen message from this sender.
  • The sender's address is appended to the "pep-challenge/responses" file so that subsequent messages from the same sender will not trigger challenges.
  • Each message listed in the index file is processed by PEP again and subjected to all the same rules. This time the challenge rules are disabled to avoid looping.
  • As each frozen message is processed, it is deleted from the "pep-challenge/messages" directory.
  • When all the messages have been processed, the index file itself is deleted as well.
  • PEP then exits and does not test the challenge response itself against your mailrule file, but simply discards it.

When a "challenge" rule is triggered, PEP looks up the sender's address in the "pep-challenge/responses" file in your home directory. If that directory/file does not exist, or if it doesn't contain the sender's address, then the following events occur:

  • The directories "pep-challenge", "pep-challenge/indexes", and "pep-challenge/messages" are created if they do not already exist.
  • The message is assigned an ID that is guaranteed to be unique.
  • The message is saved to a file in the "pep-challenge/messages" directory, using the unique ID as the filename.
  • The unique ID is appended to a file in the "pep-challenge/indexes" directory. The name for this file is yet another unique value called the "challenge ID" which is derived from the sender's address.
  • A reply file is sent back to the sender, asking him to reply in order to confirm his address.
  • PEP exits without delivering the original message.

If the sender's address is found in the "pep-challenge/responses" file, then the message is from a valid address that was previously confirmed, so PEP continues on with the rest of your mailrule file.

Note: Subsequent rules in the mailrule file may still delete the message, even though it's from a confirmed address. It all depends on how you structure your mailrule file.

Finally, files in "pep-challenge/indexes" and "pep-challenge/messages" are examined periodically and any that are too old are deleted. The default time period is 7 days, but you can override this.

How You Use It

To use the challenge system you have to do two things: create a challenge message that will be sent back to unrecognised addresses, and add one or more challenge rules to your mailrule file.

Let's start with the challenge rules. These are just like any other PEP rule except the action is "challenge". Challenge rules require one parameter, and that is the name of the reply file to send.

This example sends the file "spamchal.txt" to unrecognised senders but only if the message has a high spam score:
challenge if sascore > 5 with spamchal.txt

This example sends the file "challenge.txt" to all unrecognised senders:
challenge if * matches * with challenge.txt

You can choose to send a challenge based on any condition, but the above two examples will probably be the most common. You can have multiple challenge rules if you want, only the first one that matches the message will send a challenge.

The reply file that is sent as a challenge is just like any other, except that it must contain the following header line:

     Reply-to: you/cr-{CHALLENGEID}@christian.net

If your username is "mark", you'd use "mark/cr-{CHALLENGEID}@christian.net" instead.

Without a Reply-to: header a reply file can't be replied to at all, and in this case we need to make sure that the "/cr-" is in there along with the unique challenge ID.

NOTE: The actual username and domain for the reply address can be anything that will ultimately cause a response to pass through your mailrule file (ie: you can use your own domain name if you want). It's critical that the address contains the "/cr-{CHALLENGEID}" part though.

Here's a sample challenge reply file. Feel free to copy it or create your own:
Reply-To: you/cr-{CHALLENGEID}@christian.net
From: you@christian.net (Your Name)
To: {from}
Subject: Please confirm your message

A message you sent to me has been "frozen" because my spam filter didn't
recognise your e-mail address.

To "thaw" the message so that it will reach me, please confirm your
address by replying to this message, or send a blank message to:

you/cr-{CHALLENGEID}@christian.net

There is no need to resend your original message. You'll only have to
confirm your address this first time. Any subsequent messages from you
will be recognised and not challenged like this.

Deleting Old Frozen Messages

PEP periodically looks at all the messages that are currently waiting for a challenge response and deletes those that have been waiting too long. The default period of time is 7 days. If a response to a challenge is not received within this time, the message is deleted and you never see it.

If 7 days isn't long enough for you (or if it's too short), you can change it with the "expirechallenged" command. This command takes a single parameter which indicates a length of time. This command must come somewhere before the challenge rules to which it applies.

This example expires frozen messages after 3 days:
expirechallenged 3d

Both of these examples expire frozen messages after 2 weeks:
expirechallenged 14d
expirechallenged 2w

Hints & Tips

Avoid unnecessary challenges
Where possible, use other PEP rules to explicitly KEEP or DELETE messages based on other criteria before resorting to a CHALLENGE rule. There's no point annoying friends and mailing list admins if you don't have to! Likewise there's no point in clogging up the mail server's message queue with replies to messages that you know are spam.

For example, you might first choose to keep messages from your favourite mailing lists and from friends and family, then delete messages with a really high spam score, and then challenge everything else.

Take care when composing your challenge message
Keep in mind that some of the messages you'll be challenging are legitimate and didn't come from spammers. A well worded challenge message will prevent these people from getting confused or annoyed or offended.

Editing the challenge response file
You can edit the "pep-challenge/responses" file if you want. If you delete addresses from this file, they will no longer be recognised and the next message from them will trigger a new challenge. You can also add addresses to this file ahead of time to "pre-authorize" certain addresses.

NOTE: Entries in this file may contain wildcards, so you can "pre-authorize" all addresses from a given domain with a line like "*@aol.com".

You can delete this file if you want to reset the whole thing.