Wednesday, June 9, 2010

HOWTO - Define an SPF Record

Category: — SkyHi @ Wednesday, June 09, 2010
Additionally you will need to submit these forms.
http://www.spfwizard.com/


For yahoo: http://help.yahoo.com/l/us/yahoo/mai...r%2Findex.html

for gmail: http://mail.google.com/support/bin/r...pe=msgdelivery

for hotmail: http://postmaster.live.com/


This section defines HOWTO configure a Sender Policy Framework (SPF) record for a domain and its mail servers. DKIM/ADSP is another approach involving the signing of mail to authenticate the sender domain.

The macro feature of SPF was incorrectly documented using parenthesis, these should have been braces ({}).

SPF was initiated by Meng Weng Wong of pobox.com to enable validation of legitimate sources of email for a domain and is now an IETF standard (RFC 4408).

Briefly, the design intent of the SPF record is to allow a receiving MTA (Message Transfer Agent) to interrogate the Name Server of the domain which appears in the email (the sender) and determine if the originating IP of the mail (the source) is authorized to send mail for the sender's domain. The mail sender is required to publish an SPF RR (documented here) but the sending MTA is unchanged.

The SPF information SHOULD be defined in a standard TXT RR and MAY now be defined in an SPF RR type (BIND releases from 9.4.0 support the SPF RR type - see also RFC 4408).

If a SPF (TXT) RR exists and authorizes the source IP address the mail can be accepted by the MTA. If the SPF (TXT) RR does not authorize the IP address the mail can be bounced - it did not originate from an authorized source for the sender's domain.

Many Open Source MTAs have already been modified to use the SPF record and there is no down-side (assuming you get the SPF RR right) and plenty of potential up-side (elimination/reduction of some spam categories) to implement the SPF record now.

We use the following terminology to try and simplify the descriptions below:

  1. sender - the full email address of the originator of the mail item (typically uses return-path in the actual SPF checks)
  2. source-ip - the IP address of the SMTP server trying to send this message
  3. sender-domain the domain name part of the sender's email address e.g. assume the sender is info@example.com the sender-domain is example.com.

The SPF record defines one or more tests to carry out to verify the sender. Each test returns a condition code (pre below). The first test to pass will terminate SPF processing.

TXT RR Format

The standard TXT and SPF record format is defined as:

name  ttl  class   TXT     text
name  ttl  class   SPF     text

The SPF RR is functionally identical to a TXT record with SPF data. BIND 9.4+ supports the SPF RR type, however previous versions, and most other DNS software (as of July 2007), do not yet support the SPF RR type. Thus the RFC's recommendation is to always provide a TXT based SPF RR and, if your DNS software supports the SPF RR type, duplicate the information from the TXT version of the SPF RR in a native SPF RR. The reason for this procedure is simply because while the master/slave may support the SPF RR, querying name servers - such as name servers used by receiving MTAs - may not. Some, but not all examples, below have been updated to reflect the use of both record types to illustrate usage. In all cases the TXT and SPF RRs are shown with a comment line between containing the word AND as a reminder of the current policy recommendation. It is safe to assume for the foreseeable future that only using a TXT version of the SPF will always work.

The SPF data is entirely contained in the text field (a quoted string). SPF defines the contents of the quoted string as follows:

v=spf1 [[pre] type ] ... [mod]

Where:

v=spf1
Mandatory. Defines the version being used. Currently the only version supported is spf1.

pre
Optional (defaults to +). pre defines the code to return when a match occurs. If a test is conclusive either add + or omit (defaults to +). If a test might not be conclusive use "?" or "~" (tilde). "-"(minus) is typically only used with -all to indicate that if we have had no previous matches - fail.
Value Description
+ Default. Pass.
- Fail.
~ Softfail.
? Neutral.

type

Defines the mechanism type to use for verification of the sender. May take one of the following values:

Basic Mechanisms

These types do NOT define a verification mechanism but affect the verification sequence.

  1. include - Recurse (restart) testing using supplied domain. The sender-domain is replaced with the included domain name. Example: 
    ; spf record for example.com
    example.com.  IN TXT "v=spf1 include:example.net -all"
    ; AND
    example.com.  IN SPF "v=spf1 include:example.net -all"
    ; use the SPF details for example.net
    ; in the above case to replace example.com's SPF
    ; or
    example.com.  IN TXT "v=spf1 mx include:example.net -all"
    ; additive - use MX RR for example.com
    ; AND if that fails use example.nets's SPF
  2. all - The all type terminates processing (but may be optionally followed by a mod value). It is defined to be optional but it is a Good Thing™ to include it. It is normally present in the form -all to signify that if processing reaches this point without a prior match the result will be fail. But if you are not sure that the tests are conclusive you could use ?all which would allow mail to be accepted even if all previous checks failed.

Sender Mechanisms

These types define a verification mechanism.

  1. ip4 - use IP Version 4 addresses, for example, 192.168.3.0 for verification
  2. ip6 - use IP Version 6 addresses for verification
  3. a - use DNS A RRs for verification
  4. mx - use DNS MX RRs for verification
  5. ptr - use DNS PTR RRs for verification
  6. exists - test for existence of domain
Value Description

a
a:domain
a:domain/cidr
a/cidr

In its base form this uses the sender-domain to find an A RR(s) to verify the source. This form relies on an A RR for the domain as shown:

; fragment for example.com
$ORIGIN example.com.
example.com.  IN TXT "v=spf1 a -all"
; AND
example.com.  IN SPF "v=spf1 a -all"
; needs domain A record
@            IN   A 192.168.0.3
; functionally the same as
example.com. IN   A 192.168.0.3

The form a/cidr applies the test to the cidr (or IP refix or slash) range of the sender-domain's A RR. For more info on CIDR.

The form a:domain replaces sender-domain with domain's A RR for verification. This does NOT use domain's SPF record(s) (use include for that). The domain form may use macro-expansion features. Example:

; fragment for example.net
$ORIGIN example.net.
@            IN TXT "v=spf1 a:example.com -all"
; AND
@            IN SPF "v=spf1 a:example.com -all"
; will use a single A query to example.com
; which may not yield the result expected unless
; example.com has an A record as below
@            IN   A 192.168.0.3
; functionally the same as
example.com. IN   A 192.168.0.3

can take a host name format as shown below:

; fragment for example.net
$ORIGIN example.net.
@            IN TXT "v=spf1 a:mail.example.com -all"
; will use a single A query for mail.example.com

The form a:domain/cidr applies the cidr range to the IP address obtained from the A query e.g.

; fragment for example.net
$ORIGIN example.net.
@            IN TXT "v=spf1 a:mail.example.com/27 -all"
; AND
@            IN SPF "v=spf1 a:mail.example.com/27 -all"
; will use a single A query for mail.example.com

Any of the 32 IP addresses that contain mail.example.com will pass. e.g. if the source-ip is 192.168.0.25 and the A RR for mail.example.net is 192.168.0.2 then the test will pass.

mx
mx:domain
mx:domain/cidr
mx/cidr

This basic form without any extensions uses the MX RR of the sender-domain to verify the mail source-ip. The MX record(s) return a host name from which the A record(s) can be obtained and compared with the source-ip. The form mx/cidr applies the IP Prefix or slash range to the A RR address. For more info on CIDR. With any of the domain extensions the MX record of the designated (substituted) domain is used for verification. The domain form may use macro-expansion features.

Warning Remember the MX RR defines the receiving MTA for the domain. If this is not the same host(s) as the sending (SMTP) MTA then tests based on an mx type will fail. We have also received a report that mx on its own is rejected by certain SPF libraries. We regard this as an error and are trying to contact the library developer to clarify issues.

Examples:

; fragment for example.com
$ORIGIN example.com.
   IN  TXT "v=spf1 mx:example.net -all"
; AND
   IN  SPF "v=spf1 mx:example.net -all"
; verify sender using example.net MX and A RRs
; fragment for example.com
$ORIGIN example.com.
   IN  TXT "v=spf1 mx/26 -all"
; AND
   IN  SPF "v=spf1 mx/26 -all"
; verify sender using example.com MX and A RRs
; and use 16 IP address range
ptr
ptr/domain

Use the source-ip's PTR RR and a reverse map query. The AA RR for the host is then obtained. If this IP matches the sender-ip AND the sender-domain is the same as the domain name of the host obtained from the PTR RR then the test passes. The form ptr:domain replaces the sender-domain with domain in the final check for a valid domain name. The domain form may use macro-expansion features. The PTR record is the least preferred solution since it places a load on the IN-ADDR.ARPA (IPv4) or IPV6.ARPA reverse-map domains which generally have less capacity than the gTLD and ccTLD domains. Examples:

; fragment for example.com
$ORIGIN example.com.
@            IN TXT "v=spf1 ptr -all"
; the effect is to allow any host which is
; reverse mapped in the domain to send mail
ip4:ipv4 ip4:ipv4/cidr In its basic form defines an explicit ipv4 address to verify the mail source-ip. If the source-ip is the same as ipv4 the test passes. May optionally take the form ipv4/cidr to define a valid IP address range. For more info on CIDR. Since this type incurs the least additional load on the DNS the current draft of the proposed RFC recommends this format. Examples: 
; fragment for example.com
$ORIGIN example.com.
@      IN TXT "v=spf1 ip4:192.168.0.2 -all"
; AND
@      IN SPF "v=spf1 ip4:192.168.0.2 -all"
; if source-ip is 192.168.0.2 test passes
; cidr format
@      IN TXT "v=spf1 ip4:192.168.0.2/27 -all"
; AND
@      IN SPF "v=spf1 ip4:192.168.0.2/27 -all"
; if source-ip is in range 192.168.0.1
; to 192.168.0.31 test passes
ip6:ipv6
ip6:ipv6/cidr		 In its basic form defines an explicit ipv6 address to verify the mail source-ip. If the source-ip is the same as ipv6 the test passes. May optionally take the form ipv6/cidr to define a valid IP address range. For more info on CIDR. Since this type incurs the least additional load on the DNS the RFC recommends this format. Examples: 
; fragment for example.com
$ORIGIN example.com.
@     IN TXT "v=spf1 ip6:2001:db8::10 -all"
; AND
@     IN SPF "v=spf1 ip6:2001:db8::10 -all"
; if source-ip is 2001:db8:0:0:0:0:0:10 test passes
; cidr format
@     IN TXT "v=spf1 ip6:2001:db8::10/120 -all"
; if source-ip is in range 2001:db8:0:0:0:0:0:0
; to 2001:db8:0:0:0:0:0:FF test passes
exists:domain

The existence (any valid A RR) of the specified domain allows the test to pass. Domain may use macro-expansion features.

mod

Two optional record modifiers are defined. If present they should follow the last type directive i.e. after the all. The current values defined are as follows:

Modifier Description
redirect=domain Redirects verification to use the SPF records of the defined domain. Functionally equivalent to include but can appear on its own (without a terminating all) or can placed after the all which means "if all the previous test fail try this redirect". Examples: 
; fragment for example.com
$ORIGIN example.com.
@ IN TXT "v=spf1 ip4:192.168.0.2 -all redirect=example.net"
; if source-ip is 192.168.0.2 test passes
; if it fails redirect to example.net
; OR single redirect
@ IN TXT "v=spf1 redirect=example.net"
; AND
@ IN SPF "v=spf1 redirect=example.net"
; only use example.net SPF record
exp=txt-rr

The exp record if present should come last in a SPF record (after the all if present). It defines a DNS name whose TXT record's text may be returned with any failure message. Example:

; domain SPF record
   IN  TXT "v=spf1 mx -all exp=bad.example.com"
; AND
   IN  SPF "v=spf1 mx -all exp=bad.example.com"
; the getlost TXT record
bad IN  TXT "Not allowed to send mail for domain"

The syntax allowed by this record is significantly more complex (see macro-expansion below.

Macro-Expansion

SPF defines a number of macro-expansion features as defined below:

Note: all macro-expansion delimiters use braces {}.

Modifier Description
%{c}Only allowed in TXT records referenced by the exp field. The IP of the receiving MTA.
%{d}The current domain, normally the sender-domain %{o} but replaced by the value of any domain argument in the type above.
%{h}The domain name supplied on HELO or EHLO, normally the hostname of the sending SMTP server.
%{i}sender-ip The IP of SMTP server sending mail for user info@example.com.
%{l}replace with local part of sender e.g. if sender is info@example.com, the local part is info.
%{o}The sender-domain e.g. if email address is info@example.com the sender-domain is example.com.
%{p}The validated domain name. The name obtained using the PTR RR of the sender-ip. Use of this macro will require an additional query unless a ptr type is used.
%{r}Only allowed in TXT records referenced by the exp field. The name of the host performing the SPF check. Normally the same as the receiving MTA.
%{t}Only allowed in TXT records referenced by the exp field. Current timestamp.
%{s} Replace with sender email address, for instance, info@example.com
%{v} Replaced with "in-addr" if sender-ip is an IPv4 address and "ip6" if an IPv6 address. Used to construct reverse map strings.

The above macros may take one or more additional arguments as follows:

  1. r - Indicates reverse the order of the field, for instance, %{or} would display example.com as com.example and %{ir} would display 192.168.0.2 as 2.0.168.192. The normal split uses "." (dot) as the separator but any other character may be used to define the split but a "." (dot) is always used when rejoining so, for instance, %{sr@} would display info@example.com as example.com.info.

  2. digit - the presence of a digit (range 1 to 128) limits the number of right most elements displayed, for instance, %{d1} displays only com only from example.com but %{d5} would display five right hand elements up to the maximum available, in this case it will display example.com since that is all that is available.

Examples

Example 1

Example 1: Assumes a single mail server which both sends and receives mail for the domain.

; zone file fragment for example.com
$ORIGIN example.com.
             IN  MX 10 mail.example.com.
....
mail          IN  A     192.168.0.4
; SPF stuff
; domain SPF
example.com.  IN  TXT   "v=spf1 mx -all"
; AND
example.com.  IN  SPF   "v=spf1 mx -all"
; mail host SPF
mail          IN  TXT   "v=spf1 a -all"
; AND
mail          IN  SPF   "v=spf1 a -all"

Notes:

  1. the domain SPF is returned from a sender-domain query using the sender's email address e.g. sender = info@example.com sender-domain = example.com. The SPF record only allows the MX host to send for the domain.
  2. the mail host SPF is present in case the receiving MTA uses a reverse query to obtain the source-ip host name and then does a query for the SPF record of that host. The SPF record states that the A record of mail.example.com alone is permitted to send mail for the domain.

If the domain contains multiple MX servers the domain SPF would stay the same but each mail host should have a SPF record.

Example 2

Example 2: Assumes the domain will send mail through an offsite mail server e.g. an ISP:

; zone file fragment for example.com
$ORIGIN example.com.
            IN  MX 10  mail.offsite.com.
....
; SPF stuff
; domain SPF
example.com. IN  TXT    "v=spf1 include:offsite.com -all"
; AND
example.com. IN  SPF    "v=spf1 include:offsite.com -all"
; WARNING: offsite.com MUST have a valid SPF definition

Notes:

  1. This format should be used IF AND ONLY IF you know that offsite.com has a valid SPF configuration.
  2. include recurses (restarts) verification using the SPF records for offsite.com. Mail configuration changes are localised at offsite.com which may simplify administration.
  3. include could have been replaced with redirect.

Example 3

Example 3: Assumes we are the host for a number of virtual mail domains and that we can send mail from any host in our subnet.

Zone file fragment for one of the virtual mail domains:

; zone file fragment for vhost1.com
$ORIGIN example.com.
           IN  MX 10 mail.example.com.
....
; SPF stuff
; domain SPF
vhost1.com. IN  TXT   "v=spf1 include:example.com -all"
; AND
vhost1.com. IN  SPF   "v=spf1 include:example.com -all"

Notes:

  1. the domain SPF is returned from a sender-domain query using the sender's email e.g. sender = info@vhost1.com, sender-domain = vhost1.com. The SPF record recurses to the DOMAIN example.com for verification.

Zone file for example.com

; zone file fragment for example.com
            IN  MX 10   mail.example.com.
....
; SPF stuff
; domain SPF - any host from
; 192.168.0.1 to 192.168.0.30 (32 - bcast and mcast = 30)
; can send mail
example.com.  IN  TXT    "v=spf1 ip4:192.168.0.3/27 -all"
; AND
example.com.  IN  SPF    "v=spf1 ip4:192.168.0.3/27 -all"
; mail SPF
mail          IN  TXT    "v=spf1 ip4:192.168.0.3/27 -all"
; AND
mail          IN  SPF    "v=spf1 ip4:192.168.0.3/27 -all"

Notes:

  1. the domain SPF is returned from a sender-domain query using the sender's email e.g. sender = info@example.com sender-domain = example.com. The SPF record allows any host in the 32 address subnet which contains 192.168.0.3 to send mail for this and any host virtual domain e.g virtual1.com in the above example. NOTE: while /27 allows 32 IP addresses subnet rules remove 192.168.0.0 and 192.168.0.31 as the multicast and broadcast addresses respectively. [read more about IPv4 Classes]
  2. In the above scenario we could have used a slightly shorter version such as: 
    example.com. IN  TXT    "v=spf1 mx/27 -all"
    ; AND
    example.com. IN  SPF    "v=spf1 mx/27 -all"
    This record has the same effect as a:192.168.0.3/27 above but will cost a further DNS look up operation whereas the IP is already available.
  3. The above scenario relies on the fact that customers will only send mail via the domain example.com i.e. they will NOT send via another ISP at home or when travelling. If you are not sure if this is the case you can terminate the sequence with ?all which says kinda pass (soft fail) and let the mail go through - perhaps logging the incident to capture statistics.

If the domain contains multiple MX servers the domain SPF would stay the same but each mail host should have a SPF record.

Example 4

Example 4: Assumes that the domain never sends mail from ANY location - ever. Typically you would do this to prevent bogus mail for everyone else - it is a supreme act of self-sacrifice!

; zone file fragment for example.com
; zone does NOT contain MX record(s)
...
; SPF stuff
; domain SPF
example.com. IN  TXT   "v=spf1 -all"
; AND
example.com. IN  SPF   "v=spf1 -all"

Notes:

  1. This SPF test will always fail since the only condition it tests is the all which results in a fail.

Example 5

Example 5: Illustrates various macro expansion features:

; zone file fragment for example.com
$ORIGIN example.org.
           IN  MX 10 mail.example.com.
....
; SPF records
; domain SPF
@       IN  TXT   "v=spf1 exists:%{ir}.%{v}.arpa -all exp=badguy.example.com"
; AND
@       IN  SPF   "v=spf1 exists:%{ir}.%{v}.arpa -all exp=badguy.example.com"
badguy  IN  TXT  "The email from %{s} using SMTP server at %{i}
                    was rejected by %{c} (%{r}) at %{t} because it failed
                    the SPF records check for the domain %{p}.
                    Please visit http://abuse.example.com/badguys.html
                    for more information"

Notes:

  1. The badguy TXT above is split across multiple lines for presentation reasons only and should appear on a single line in the zone file.
  2. The exists:%{ir}.%{v}.arpa test is a great example BUT IT WILL NOT WORK because the exists type checks for an A RR whereas a reverse lookup is defined using a PTR RR. But it does show the power of macro-expansion and we could not think of a better one. However Stuart Gatham could, and suggested using the reversed IP address in a DNS Black List (DNSBL) as shown here: 
    @       IN  TXT   "v=spf1 exists:%{ir}.blacklist.example.com -all exp=badguy.example.com"
    ; AND
    @       IN  SPF   "v=spf1 exists:%{ir}.blacklist.example.com -all exp=badguy.example.com"
    You may also want to change the text in the badguy.example.com record to reflect the new failure. Many thanks for the suggestion.
REFERENCES
http://www.zytrax.com/books/dns/ch9/spf.html