Wednesday, March 31, 2010

Recommended DNS SOA record TTL default stackoverflow.com?

SkyHi @ Wednesday, March 31, 2010
We currently have our DNS SOA record set to the following for stackoverflow.com:

primary name server = ns1.p19.dynect.net
serial = 2009090909
refresh = 3600 (1 hour)
retry = 600 (10 mins)
expire = 604800 (7 days)
default TTL = 60 (1 min)

Are there better choices for our refresh / retry / expire / default TTL for a site like stackoverflow.com which receives close to 1M pageviews per day?


The actual traffic rate to the site is irrelevant.
All of those settings (except for "default TTL") only affect how frequently your domain's secondary DNS servers poll the primary DNS server for updates.
If your zone only changes infrequently (which I believe yours does) then your value for "refresh" is currently a bit on the low side. Typically the primary should send a NOTIFY message to each of the secondaries whenever there's an update at which point the secondaries grab the zone file immediately. These days the "refresh / retry / expire" mechanism is only a backstop to that.
In any event, it's likely that your DNS provider is automatically syncing changes to all of the relevant DNS servers on the fly without using DNS's built-in synchronisation mechanisms so the actual values are probably irrelevant.
Note that the "default TTL" field no longer means what it says. The real default TTL is set (in BIND at least) with the $TTL directive, and that's only used when there isn't an explicit TTL set on each record.
The "default TTL" field's meaning was changed in RFC 2308 and it's actually a hint for negative caching. If your server returns a negative response (e.g. NXDOMAIN or NODATA) it's how long the remote server should wait before trying again.
The current value is a bit on the low side, but there's no harm leaving it as is. It's often ignored anyway.


Interestingly, the DNS diagnostic page from the dyn guys (our DNS hosts)..
http://dnscog.com/report/stackoverflow.com
.. says this on MINTTL:
Check SOA MINTTL
Your SOA minttl value is 60 seconds, which is lower than the recommended minimum for general DNS use. If you regularly make changes to your DNS zone, or use DNS-based load balancing services, a small value here is OK.
Recommendation
Consider putting value between 1800 and 86400 to your SOA minttl field.
and this on SOA refresh
Check SOA refresh
Your SOA refresh field is 3600 seconds, which is lower than the recommended minimum. Having a low refresh value can result in unnecessary query volume or unexpected behavior, especially if you use a value of 0. If you regularly make changes to your DNS zone, or use DNS-based load balancing services, a smaller value will help to ensure changes propagate as quickly as possible.
Recommendation
Consider putting value between 7200 and 10800 to your SOA refresh field.
Another diagnostic page at http://www.intodns.com/stackoverflow.com doesn't offer any real hints.




Setting Up MX Records In Your DNS Information

The process of updating DNS with correct MX records for your domain seems to be a source of problems and confusion, so I've tried to provide some information here that may help you avoid some of the common pitfalls.

The Basics

MX records in a DNS "zone file" have the following structure:
destination.domain.
TTL
IN
MX
preference-number   mail.server.name.

Where:
destination.domain.
The host or domain name that the MX record applies to.  This name appears after the "@" in an email address, and is normally your domain name ("foo.com") and not the name of your mail server ("mail.foo.com").  Note that if you use the full domain name here, it must have a trailing dot (".").
TTL
The DNS "time to live" value for this record.  Since your MX records don't change very much, a reasonable value for this field is 12 hours (specified as "12h" or "43200" in seconds).  Note that if your mail server uses a dynamic IP address then the DNS "A" (address) record for it should have a short TTL (60 seconds is common), but your MX records can still have a long TTL.  This field may be omitted, which case the default TTL for the zone applies.
IN
Specifies that this record applies to the "internet" class of names.  This field is commonly omitted; it defaults to the type of the previous field, which is almost always "IN".
MX
The type of DNS record this is.  MX == "Mail eXchanger".
preference-number
A numeric value that indicates the preference that should be given to this MX record relative to other MX records that apply to the same destination domain.  Lower numbers imply higher preference.
mail.server.name.
The hostname (not IP address) of a mail server that will accept mail addressed to "user@destination.domain".  If you use a fully qualified domain name here (which you must do if you're using my servers), then the name must have a trailing dot.  Note: only list a mail server here if it listens on the standard SMTP port (25).

For example, suppose that I've registered the domain name "example.com".  I've set up a Linux machine as my mail server and called it "smtp.example.com".  My ISP blocks port 25, so my server only listens on port 2525 and I can't list it as an MX record in my DNS information.  Instead, I've signed up for domainMX.net service, so I'm listing two of the domainMX.net servers in my DNS info and they will forward my mail to me on port 2525.
My DNS information should look something like this (I've left out the optional TTL and class fields):


example.com.
MX
10  mx1.domainMX.net.

example.com.
MX
20  mx3.domainMX.net.

Note the trailing dots on all fully qualified names.  Note as well that the name of my mail server (smtp.example.com) doesn't appears anywhere in this information: that's what we want, since the only servers that need to know about my mail server are mx1 and mx3.
If I switched ISPs to one that doesn't block port 25, I could update my mail server to listen on the standard port and add a third line to my MX information:


example.com.
MX
5   mail.example.com.

example.com.
MX
10  mx1.domainMX.net.

example.com.
MX
20  mx3.domainMX.net.



Other Issues

There are some other common issues that people run into:

MX records pointing to a CNAME

The hostname on the right side of an MX record (the "mail server name") must exist in DNS as an "A" record and not as a CNAME (alias) record.  For example, the following DNS information is incorrect:


foo.example.com.
A
192.168.0.10


mail.example.com.
CNAME
foo.example.com.


example.com.
MX
10   mail.example.com.
; WRONG: "mail" is a CNAME

However, you can do something similar by creating a second "A" record for "mail.example.com":


foo.example.com.
A
192.168.0.10


mail.example.com.
A
192.168.0.10


example.com.
MX
10   mail.example.com.
; OK

Don't create your own hostnames for my servers

Several people have asked if it's okay to create hostnames in their domain that refer to my server by IP address:


mymx1.example.com.
A
209.61.212.15
; this IP is mx4.domainMX.net

example.com.
MX
10   mymx1.example.com.
; Not Recommended!

I strongly recommend against creating your own hostnames for my servers.  The IP addresses associated with my servers do change, and more frequently than you might expect: load balancing, ISP renumberings and server moves all result in such changes.  If you refer to my servers using the hostnames that I provide then these changes will be transparent.  Otherwise you run the risk of delayed or even lost mail.

Mail to subdomains doesn't work

Consider the following DNS information:


server.example.com.
A
192.168.0.1

mail.example.com.
A
192.168.0.5

example.com.
MX
10  mx1.domainMX.net.

example.com.
MX
20  mx3.domainMX.net.

Mail sent to "user@example.com" will be delivered to mx1.domainMX.net, or if mx1 is down, to mx3.domainMX.net.  From there it will be forwarded to my mail server on "mail.example.com" (this forwarding is configured on mx1 and mx3, not in DNS).
However, if mail is sent to "user@server.example.com" the two MX records above are ignored and delivery is attempted directly to "server.example.com".  If there is no SMTP server listening on that host, the mail will be returned to sender.  Why?  MX records are used only when the entire "host part" of an email address (the part after the "@") matches the "destination domain" in the MX record.
So, in this example "server.example.com" is not an exact match for "example.com", so the MX records are ignored.  If you want mail for "server.example.com" to also be delivered to the same domainMX.net servers, you need more MX records:


server.example.com.
MX
10  mx1.domainMX.net.

server.example.com.
MX
20  mx3.domainMX.net.

Note: if you set up MX records for a subdomain that point to domainMX.net servers, you must let me know so I can update my configuration.  By default my servers will only deliver mail addressed to the domain itself, not subdomains.  There's no extra charge for this.

What about wildcard MX records?

A wildcard MX record is one where the "destination domain" starts with "*", as in "*.example.com".  An MX record like this means "send mail addressed to any host under example.com to this MX server".  My experience with these is that they're usually more trouble than they're worth, and I'd recommend you consult a good DNS resource and understand the drawbacks before you use them.

How many DNS servers do I need?

I strongly recommend that you have at least two separate DNS servers for your domain.  They should be reliable, well-connected servers located on different subnets and preferably in different cities.  Otherwise you may encounter a situation where none of your DNS servers are reachable, which may cause mail to (or even from) your domain to be rejected because your domain appears invalid.
BackupDNS.com has a more detailed explanation of why a solid set of nameservers is important.
I do not provide DNS services, but I'd added a few links to sites that do on my links page.

Where can I get more information about MX records or DNS in general?

I highly recommend the O'Reilly book DNS and BIND.  It's available both as a print version (mine is well used and never far from my computer) and on-line as well at the O'Reilly site. 

REFERENCES

http://serverfault.com/questions/69183/recommended-dns-soa-record-ttl-default
http://domainmx.net/mxsetup.shtml