Command Center

As an email administrator you will be challenged with creating filters, blacklist, whitelist, and/or redirects that require an understanding of email headers. In their simplest form email headers are read by an email client to display the To, From, Date, and Subject of a message.

Message headers (From: and To:) differ from senders and recipients and I'd like to outline those differences so when you are confronted with creating filters you don't end up pulling your hair out trying to figure out why a simple filter does not work. :)

1. Why doesn't my filter work?
2. Headers and SMTP Envelope
3. Read Email Headers
4. Senders and Recipients

Why doesn't my filter work?

The most common problem I see when asked why a filter does not work is in the differences between the From: header and the sender. Here's what usually happens. Worker receives an email they no longer want to receive from a certain email address. They ask you to block email address user@spam_domain.com. So you create a filter to reject user@spam_domain.com. The next day rolls around and your filter is not working according to the worker. Why? The email reader (Outlook, Thunderbird, etc...) reads the From: header and not the Sender and the mail server filter you created is likely triggered on the sender address. Lets explain how this is possible.

Headers and SMTP Envelope.

A good analogy of what is happening is in a written letter that is addressed to you on an envelope, yes I'm talking snail mail here..., but the letter inside the envelope is addressed to and from someone else. When an email is received by your mail server it takes the letter out of the envelope and puts it in your inbox. Your email address may not be visible in the To: header and the original sender (or return address) may not be the same as the From: header. This is all perfectly legal in the email world.

The SMTP envelope of the message will always contain the actual sender and recipient(s) of a message and you can view this from the SMTP logs of your mail server.

The From: and To: headers are sent during the DATA command of the SMTP session and is viewed by the email reader (Outlook, Thunderbird, etc...) when the message is opened.

Since SMTP has this ability it's able do things like mailing list and BCC. A mailing list will typically put the mailing list email address in the To: header, yet it still arrives in your inbox without you seeing your email address in the To: header. Probably the best example is BCC. When you BCC someone the BCC recipient is not included in the headers of the message only as a recipient. And this is all controlled by your email reader once you press the Send button.

Reading Email Headers.

After you press the Send button in your email client it has to create the message, and decide who the sender and recipients are. The creation of the message includes email headers, the body of the message, and any attachments. I'm not getting into how attachments are created as it involves explaining mime boundary headers and that's out of the scope of this article.

Email Header example:

Subject: Weekly Report Update
To: cindy@domain.com, joe@domain.com
From: fred@domain.com
Date: Fri, 01 May 2009 10:08:12 -0400
X-headers: Optional information - Such as, Thunderbird 2.x

Body of message
.
The above example is in it's simplest form, you will see many other headers in an email message, all of which are usually self explanatory.

Note, the "." on a line by itself just below the 'Body of the message'. The "." on a line by itself is only needed during the SMTP session to tell the mail server that it's received all the DATA and can save the message for delivery. When the mail server saves a message in your inbox it may not include the "." as it's not required by an email client to read the message.

The above example only shows what an email client will create after sending a message. The mail server will also add headers to the message or can modify headers as needed. Here's an example after a message passed from an email client through two mail servers.

Received: From Domain.com (123.123.123.123) to remote_mail.com
Received: From Email client (192.168.1.5) to local Mail Server Domain.com
Subject: Weekly Report Update
To: cindy@domain.com, joe@domain.com
From: fred@domain.com
Date: Fri, 01 May 2009 10:08:12 -0400
MIME-Version: 1.0
X-headers: Optional information - Such as, Thunderbird 2.x

Body of message

Each time a message is passed from one mail server to another a Received: header is added to the top of the message identifying the mail server that delivered the message. Some email clients do not show the received header when you use the option to view headers. Ideally to see all headers you should view the message file on the mail server. Use a text editor to view the message.

When creating filters you can parse Received: headers but only if the Received: header exist, meaning it's already passed through one mail server. I've seen cases where mail admins try to filter on the Received: header of itself and usually that's not possible.

Message Header Formats.

The format of the email headers are critical. The order of the headers are not critical, other than the Received: header, which is always placed at the top of the message by the last mail server that delivered it.

In order for an email header to be consider an email header it must have a colon at the end and it must be before the message body. The header section of the message will always have one header on each line. A blank line or enter starts the body of the message.


Senders and Recipients.

The SMTP protocol has some flexibility that is not always obvious on the surface. Always consider the sender, recipient, From:, and To: headers when creating filters and you should save yourself some troubleshooting steps later on.

REFERENCES
http://www.mailserverblog.com/2009/05/how-to-read-email-headers.html