Command Center: Exim, or Postfix, or Qmail, or Sendmail

Tuesday, March 30, 2010

Exim, or Postfix, or Qmail, or Sendmail :: MTAs Comparison

Category: Postfix, Sendmail — SkyHi @ Tuesday, March 30, 2010

This article was adapted from www.lwn.net -- published in August, 2006 with many contributors on LWN and since then.

Contents

For a lot of people the choice of the Mail Transfer Agent
is important. A bad choice can mean lost time and money, lower
reliability and increased risk to networks. A good choice can,
depending on the architecture of your mail system, remain substantially
unchanged for years. If you're designing a major mail system then
you're also going to be asking yourself the question should I use more than one kind of MTA, and if so which should go where? Hopefully this article can help.

Too busy? Don't care about the pros and cons? Here is the Quick Answer.

I only cover MTAs related to Unix/Linux

Debates over MTAs sometimes last for years, and this article covers
the main points that come up over and over. Unfortunately, apart from
this article there are no general comparisons of MTA characteristics on
the Internet, and even very little benchmarking. The remarks here are
personal opinions drawn from readily-verifiable facts and subjective
comments drawn from experience. Nearly every MTA has a vociferous and
sometimes combative group of supporters, not always including the
principal authors of the MTA.

It is easy to see why administrators care about which MTA they use.
Large installations require a lot of time spent tuning the MTA, and for
any site email is commonly regarded as the most important use of the
Internet.^[1]
End users can get by without a web site or a browser for a little, but
without email business stops. And so countless administrators invest
time in learning how to tweak their internet mail delivery tool in
order to meet their various goals. But which tool should they use when?

As at the beginning of 2007, most Internet email seems ^[2] to be delivered by one of four MTAs:

Postfix
Exim
qmail
Sendmail

There are other worthy free MTAs to talk about, such as zmailer and
smail3, but since they are not so widely used I omit them. There are
some MTAs that are a small component of a larger system such as
Courier-MTA (despite the name, which doesn't fit any definition of MTA
used here!) which it isn't reasonable to compare. There are some
unworthy MTAs too, these I am delighted to omit. There are also changes
afoot I expect to cover here soon, including the rise of blackbox
front-end email systems and new MTAs making their debut.

HowTo Compare MTAs

Current and recent versions (say, since 2002) of each of these four
widely-used MTAs have broadly similar features. All of them have, on
the basis of a very large body of evidence ^[3], the following characteristics:

a good security record (at least good; some have an excellent record)
ability to handle very large amounts of mail
interact with databases in many formats
can speak many of the SMTP

variants in use
source code is available in a free manner
quality third-party documentation is available
there are significant user communities

There are some assumptions implicit in the rest of this article.
This document is not for you if you are looking for a product that:

address
presents
a comprehensive GUI administrative interface as a major selling point.
Such GUIs exist for most MTAs considered here, but they are by no means
a selling point. If you're making a choice on the basis of the
administrative GUI (as opposed to range of administrative functions,
and how easily and efficiently they can be used) you probably need an
email consultant.

Lotus Notes and Microsoft Exchange are examples of systems that
don't meet the criteria for this document. They are able function as
MTAs, but that is only a small part of what they were designed to do.

No MTA scores well by all measures. The needs of users vary greatly
and some criteria are mutually orthogonal. Commonly cited MTA selection
criteria are:

Ease of administration
Security
Performance
Long-term viability

Design features partly decide how much each MTA meets these
criteria, with the rest coming from where the development team have
interests. Contradictory examples of these features are:

single configuration file, so everything is in one place
many single-purpose and optional configuration files
minimal and careful syntax
powerful embedded scripting language
maximum code stability
source code contributions regularly incorporated
minimum possible features added

These features are all valid. Which features you need depends on what you're looking for.

Just about every mail delivery scenario can be met, in one way or another, by all four MTAs. There is no one right answer.

Surveys

There is no equivalent of the Netcraft Web Survey
for SMTP servers. SMTP surveying is a very different task to what
Netcraft (and other companies) do for the web, but not particularly
harder. It is astonishing how little definitive information there is
about what SMTP servers are in use on the public Internet.

I picked the Big Four MTAs for this article based on discussions
with the people who run busy mail hubs in public ISPs, by lurking in
the email community and from taking small samples of what is running
behind the world's MX records. I also take into account that many
administrators do not change the MTA that comes with their Unix-like
operating system. No major Linux distribution ship Sendmail by default,
except for Red Hat prior to 2007^[4], and only Sendmail on commercial Unix and some BSD platforms. Indicators such as the Debian Popularity Contest are flawed as statistical measures although they can give a flavour of what is happening in one corner of the Internet.

In January 2007 an article was published with the hopeful title Fingerprinting the World's Mail Servers
and it illustrates that SMTP surveys are not so easy. Despite making
what appears to be a very competent effort the authors settle for
sampling 400k domains belonging to companies listed by a US data firm.
While interesting, this data clearly cannot tell us which SMTP servers
are run on the Internet. I have asked Ken Simpson and Stas Bekman if
they would mind sharing their detection code for incorporation into a
more complete survey design.

Dan Bernstein's 2001 SMTP survey
of one million hosts put Sendmail at about 42% market share, but the
methodology described is unlikely to give representative results for
the Internet as a whole. Like the Simpson/Bekman article there is only
an overview description of the approach, so it cannot be replicated.

Qmail

Qmail

MTA details
Website:	http://www.qmail.org
Out since:	1996
Goals:	Security; Simplicity; Efficiency
Non-goals:	Unix conventions, ease of admin
Licence:	not open source or free
Classification
Config:	Many simple control files
Releases:	Never (since 1997!)
Committers:	1
Maj. contribs:	0
Flexibility:	Very, if you study hard
Subjective Comments
Administration:	Buy one of the books!
Security:	Good record
Performance:	Excellent
Community:	Smallish but very active
Sendmail compat:	Good

Note: qmail is unmaintained: the author has not released
since 1997 and does not permit others to make releases. It is also not
Open Source Software, although the source is visible and usable within
very tight restrictions.

So why should anyone care about qmail? Perhaps they shouldn't in
2007, but there were good reasons to notice qmail in its first five
years of life, and there is still considerable publicity surrounding
it, and some very happy users:

qmail had a radical and seemingly impregnable ^[5] security design.
qmail solved in one stroke all the problems of the hideous BSD mailbox format with the Maildir message format.
qmail
was fast. If your only other choices were Sendmail or smail, qmail was
a welcome alternative. In those days a lot of high-volume list
management was still done on IBM mainframes; qmail was one of the
earliest serious alternatives.

These days these advantages are at least equalled by other MTAs, and
Maildir has become Maildir++. Yes, qmail taught MTA users and
developers some lessons. No, qmail isn't reagarded as a realistic
option these days: it doesn't support modern mail standards or even
IPv6; it isn't maintained; it isn't possible for someone else to
maintain it; its many oddities are increasingly painful relative to any
benefits qmail has.

In 2007 a qmail maestro is someone who knows which collection of
patches to apply to a ten year-old program. In 2004 a group of qmail
experts put together netqmail,
a distribution of qmail patches. But even that hasn't been touched for
years, and now there are patches for the netqmail patch collection.
Then there was qmailrocks which claims to be superceded (with a non-trivial upgrade path) by the author of the patches used by qmailrocks. Someone has pointed me to qmail-ldap as being used by ISPs. As observed at the beginning, you can
use qmail to build a significant mail hub. But with the requirement to
negotiate the chaos of qmail patch sets, you need to have a very
special brand of dedication to ensure your mail hub is a good one. You
might find qmail works for small sites without much configuration, but
that will entirely depend on how your particular email workload is
skewed -- for example, you might suddenly become invisible to some of
your customers due to backscatter spam.

qmail is still used on some very high-volume sites, and there are
still people who strongly believe that qmail is very correct code.
qmail source is legal to copy, use and patch. The author's licencing analysis is thought-provoking but thus far irrelevant to the global copyright debate.

qmail comes with more free personality than nearly any other program
-- what other MTA is likely to ask "hath the daemon spawn no fire?"
when it can't start?

Postfix

Postfix

MTA details
Website:	http://www.postfix.org
Out since:	1997
Goals:	Security; Easy of use; Standards
Non-goals:	General purpose MTA
Licence:	IPL (a disused license)
Classification
Config:	Single control file
Releases:	Infrequent
Committers:	1
Maj. contribs:	3
Flexibility:	Easy to change
Subjective Comments
Administration:	Intermediate, good docs
Security:	Good record. Credible team.
Performance:	Excellent
Community:	Medium-sized
Sendmail compat:	Very good

Design goals: Secure, easy to administer, efficient.

Postfix is, like qmail, written by a prolific Unix security software author, this time Wietse Venema
although unlike qmail the result is a suite with an interface like most
other Unix programs. Postfix is almost entirely the work of Wietse,
with occasional contributions in isolated areas such as integrating the
Transport Layer Security (TLS) libraries. Releases come in bursts, with
some releases containing only very small improvements. Release
management is by Wietse personally.

Postfix has a monolithic main configuration file like Exim and Sendmail^[6].
It is table-driven, everything is a table and a table can be
represented in all kinds of ways from plain text files to databases to
relational databases and more. It handles regular expressions in many
contexts, using the Perl

Compatible Regular Expression library developed by Phil Hazel for Exim. Postfix consists of about 150k lines of code.

Postfix fits somewhere between qmail and Exim. It consists of
several programs (but fewer than qmail), and has a monolithic
configuration file. Postfix has a strong emphasis on security, but not
to the extent of imposing unusual Unix management practices. Postfix is
quite flexible in its configuration file, but not to the extent of
Exim. Postfix postdates qmail and follows a vaguely analogous security
approach, an approach which was relatively more important in 1997. The
Sendmail team explicitly acknowledge Postfix as an influence on their next-generation replacement for Sendmail, MeTA1.

Postfix is less versatile than Exim, and this is largely due to its
foremost design criteria being security. Wietse specialises in writing
software that is demonstrably harder to break, and that design goal
prohibits some very convenient features that Exim can offer. Postfix is
further towards the paranoia end of the security spectrum than
Exim or Sendmail. The impressive achievement is that Postfix delivers
such great flexibility and ease of administration despite meeting
strict security goals. qmail is also regarded as highly secure, however
it does not, by design policy, give the administrator anything like the
flexibility of Postfix. If you have good reason to be paranoid, these
sorts of considerations may mean that in a mail architecture you use
Postfix for Internet-facing services and something else behind it.

Postfix has been measured by many as being extremely fast, and I
have found it very efficient. My impression is that it is more
efficient than Exim but not to a noticeable on modern hardware degree
even with very high load. Postfix and qmail seem to use about the same
amount of memory but by deliberate design qmail uses more bandwidth

Postfix is, like Exim, a drop-in replacement for Sendmail. Besides
just implementing the sendmail command line interface, Postfix is
compatible with Sendmail milters, an impressive and unequalled achievement to those that have investment in such modules.

The Postfix community is very active. Online documentation is quite
good but scattered. There are three Postfix books in English.

See http://www.postfix.org and http://postfixwiki.org.

Sendmail

Sendmail

MTA details
Website:	http://www.sendmail.org
Out since:	1982
Goals:	Be backwards-compatible
Non-goals:	Best practice
Licence:	Bespoke Open Source
Classification
Config:	Single control file
Releases:	Regular
Committers:	many
Maj. contribs:	many
Flexibility:	Enormous, but complex
Subjective Comments
Administration:	Hard to do well
Security:	Terrible. Better now.
Performance:	Ok for many
Community:	Large
Sendmail compat:	Bug for bug :-)

Design goals: Current Sendmail must be backwards-compatible. (The forthcoming MeTA1 project, formerly Sendmail X, is a rewrite and has design goals similar to Postfix.)

Sendmail 8 consists of about 118k lines of code, but that does not
count the functionality in the M4 scripts used to generate the config
file, nor milters. Documentation is good, and uniquely among MTAs there is a dominant company dedicated to Sendmail services. The Sendmail Consortium is dedicated to maintaining the Sendmail codebase.

Sendmail has an extraordinarily obscure configuration file, a poor
history of security breaches and a design centred around Unix in the
early 1980s. Sendmail is known for being inefficient compared to
alternatives so it might be hard to see why sendmail is still used at
all, but history has its own inertia. There is no good reason for a
site without Sendmail experience to install it, given the effectiveness
of the alternatives. There are often good reasons why a site with
Sendmail working should keep it.

Despite all this, Sendmail:

has improved greatly in security and performance since about 2000, and has a large number of new features
is installed by default on most commercial Unix operating systems (but few Linux distributions)
works with little or no modification with the default settings
has
a large following of systems administrators who have battled with and
now understand to some extent how to configure and run it
is a well-known MTA name, see previous comment about inertia

Sendmail has been ported to many systems, including some that are
not Unix-like such as Windows. Postfix isn't realistically portable to
Windows, and Exim is something of a second-class citizen on Windows
since it runs via Cygwin. So portability might be a reason to run
Sendmail.

Exim

Exim

MTA details
Website:	http://www.exim.org
Out since:	1995
Goals:	General purpose MTA
Non-goals:	Security
Licence:	GPL
Classification
Config:	Single control file
Releases:	Regular
Committers:	1
Maj. contribs:	many
Flexibility:	Enormous
Subjective Comments
Administration:	Straightforward
Security:	Quite good
Performance:	Very good
Community:	Large
Sendmail compat:	Very good

Design goal: General-purpose MTA for Unix machines.

Exim was inspired by the author's work with the smail 3 source code,
which was itself provoked by the many problems of sendmail. So Exim too
is a Sendmail drop-in replacement.

The outstanding feature of Exim is the intention that it be a
general-purpose mailer. Exim is not a total rethink about how mail
works, like qmail is. Nor does it restrict its feature set in order to
achieve theoretical security, like Postfix. Exim instead tries to give
administrators what they asked for, with of course a strong interest in
security, reliability and performance. Exim cannot ever be considered
as secure as Postfix, but then, it seems to be quite secure enough for
most normal applications. Of the many compromised Unix machines I've
met quite a few have been running Exim but there was no suspicion that
Exim was the cause of the breach. Think about this carefully, a lot of
people mistakenly view security as an absolute where it is instead a
continuum^[7].
In exchange for a lower level of design security -- for example, a
willingness to cooperate with unknownable external programs for
handling messages throughout message processing -- you get versatility
Postfix cannot deliver. The counterpart to this paragraph is in the
Postfix section. As a historical footnote, Exim3 had some flaws and
consequently a couple of serious security breaches. Exim4 has a
different design and has not had serious issues.

Exim behaves much like any traditional Unix daemon, with a
monolithic configuration file, a monolithic daemon, small number of log
files and a standard style of spooling. It has a very good security
record over the last seven years (early releases had classic security
issues), can cope with high load, and it has excellent integration
facilities. Exim can be extended in many ways - it is even possible to
compile in the entire perl

Exim is used at some very high-volume sites where it provides good
service, so performance comparisons saying qmail and Postfix are faster
and handle queuing better don't necessarily have any bearing on
real-world conditions (in 2007 on current hardware and with current
definitions of high load.)

Afterwords

Using More than One MTA

A split MTA architecture is often useful. For example, separate MTAs for each of:

receiving mail from the Internet, and first level of anti-spam/malware
processing mail, ie second level of anti-spam/malware and site-specific processing
delivering mail locally, ie saving to whatever mailstore solution you use (Maildir, IMAP, etc)
sending mail to the Internet

There are efficiencies to using just one MTA, including in human
resources and potentially reduced human error. Equally there can be
good reasons to choose different MTAs for different purposes.

One scenario is to use Postfix and Exim. The logic for Postfix is
that an MTA facing the Internet needs to be as secure as possible it
must perform as few functions as possible. For Exim, an MTA used to
maximise the quality of email in a second level of anti-spam/malware
must be as flexible as possible.

Where there can be disc I/O performance problems, another scenario
is to use just the delivery portion of Postfix for local deliveries
because it is relatively small and fast, and Exim for everything else.

For sites already running Sendmail but experiencing performance
problems it can help to leave all the routing logic to Sendmail (with
the configuration files developed over many years) but to do the
intensive spam/malware processing with either Exim or Postfix.

Open Source at Work

One of the interesting things about the three non-Sendmail MTAs here is the ideas and code that are shared. Postfix uses the Perl

When Local Security Isn't a Problem

The reason - not the biggest but the only - reason why Unix MTAs
have to work so hard at security is because of the Unix tradition of
local delivery to files owned by an individual user. There is a kind of
collective Unix racial memory relating MTAs to high-risk operations.
Anything listening on port 25 must have root privilege because it is a
privileged port, but all kinds of other servers have the same issue
including web servers.

The MTA mixture of setuid binaries, specially-owned directories,
pedantic authentication of local destinations, paranoia over filesystem
access is all to do with having the MTA write to a file owned by some
other user, usually by impersonating that user. Of course that is
fraught with potential danger, and no matter how well the code is
written a careless administrator can still make the entire process
unsafe by changing file permissions. It can be difficult to impose
modern partitioning security schemes on top of traditional Unix local
delivery too.

But in millions of sites this is no longer an issue, because mail is kept in a central (usually IMAP-accessible) mailstore until the user chooses to view it. Mail comes into the SMTP daemon, which then makes an LMTP delivery to the IMAP daemon. Nothing touches local files. You can run the daemon as user nobody.
It is possible to configure at least three of these mailers so that
none of the potentially dangerous code is even on your system. Here's
how:

Postfix.
Exim. All routers,
directors, and transports are compiled only when specified in the
Local/Makefile. You can compile Exim with only the smtp transport - and
make that use LMTP to 127.0.0.1 for "local" delivery if you want. Then
you can run Exim entirely in "unprivileged" mode, where it runs as "exim" the entire time, except when starting up the listening daemon.
qmail.

Quick Answer

The answer is Exim. If Exim doesn't do it for you then look at the pros and cons and don't read this section :-)

Exim can solve any MTA problem at least well and often excellently,
has very good documentation and a most supportive community. It is the
only modern mailer to expressly aim to be general-purpose. That is why
it is always the best default recommendation. There are no ordinary
circumstances where Exim is a bad choice, although there are
circumstances where something else will clearly be better.

Think of Exim as the Linux of free MTAs. There are many free
Operating Systems and some of them are better than Linux for specific
tasks. But Linux can do (at least) a good job for nearly everyone^[8].

This table is a bit less of a blanket statement:

MTA Suitability from 0 (bad) to 3 (good)
if you are...	qmail	Exim	Sendmail	Postfix	Notes
Inexperienced	0	3	1	3	Exim and Postfix have good docs and clear examples
Worried about security	3	2	0	3	Postfix is secure and modern; qmail is secure but very old and cranky; Exim is secure to different criteria (read above.)
Relying on Sendmail milters	0	1	3	2	Postfix can run milters; can use equivalent Exim routers/filter script
Wanting minimum hassle	0	3	0	3	Sendmail has some easy front-ends, but the deeper you go the worse it gets. Postfix and Exim are more predictable.
Resource-constrained	3	2	1	2	See Embedded Application below for other comments
On Windows	0	2	3	0	Sendmail has a native Windows port; Exim is in the Cygwin distro
Needing commercial support	1	3	3	3	There are competent companies for all MTAs; qmail is inherently less supportable being so old

Embedded Applications

This article is not about embedded MTAs, mail servers running
on tiny boxes, mobile phones etc. However several commentators have
taken exception to my weightings under Resource Constrained. It isn't as simple as you might think, and these weightings can only be illustrative. Here are points to consider:

resources means more than memory.
Sometimes having just one process in memory is better than several that
get pulled in as needed, even if the total amount of memory used is
greater, because questions arise such as "pulled in from where?" and
"at what cost to do the pulling?". For a long-running embedded device
with resource constraints that needs a full-featured MTA chances are
Exim or Sendmail will be more suitable than Postfix or Sendmail. If
you're using XIP (Execute In Place) this changes things again.
Embedded
devices are designed according to specific criteria, not general ones.
So you may be looking for a small and fast MTA, or a small and
full-featured MTA, or an MTA that can boot very quickly. Postfix and
qmail can be ready to receive before the rest of the system has even
finished booting because they have a very minimal and separate
receiving process that doesn't have to wait for other libraries to be
initialized. These days a Unix computer can fit inside an RJ45 socket
Exim
can be stripped down by not including some features at build time. It
still isn't tiny, but it certainly doesn't represent the code size you
see if you use say size `which exim` on your Unix
machine. On the other hand, if the task to be solved is very limited in
scope, you may be able to use just a few programs from Postfix,
probably smaller than the most stripped-down Exim you can build.
qmail
has licensing problems. Read the fine print, but you'll probably need
explicit permission to ship qmail in an appliance. Remember, it isn't
Open Source or even close.

Some Home Truths

Sendmail can be made to do anything but is
for people with a Sendmail background. It makes little sense for people
who don't have a specific need for Sendmail to learn it. The beauty of
this recommendation is that if everyone follows it, Sendmail will be
dead in a generation. Good. The Sendmail developers have started again
drawing from Postfix and qmail. Good again.
qmail is a
specialist product with a lot of drawbacks in general use. qmail
requires a very substantial commitment to master. Unless you have a
good reason to use it, don't. A hunch that qmail is more secure is not
a good reason, for most normal purposes Postfix and Exim are just as
secure. The usage terms (there isn't a licence, it is worth reading
why) is a serious isuse for longevity considerations.
Postfix is very flexible, but is also is deliberately limited by its design ^[9]
and has a tiny development community (not to be confused with its large
user community.) So it has a less predictable future. The IBM licence
has been abandoned by IBM, and precludes sharing with GPL code due to a
GPL3-like view of software patents.
None of these points are personal opinion.

Footnotes

↑ Possibly changing fast since in 2007 many young people almost exclusively use instant messaging or similar communication models
↑ I.E. seems to me personally, which is why I have included a section on surveys that also talks about my subjective impressions!
↑
Evidence of the sort that says "here's a list of high-volume mail hubs
running MTA x, so it can certainly be done", rather than a rigorous
survey -- did I mention there is a section on surveys? This evidence
can be seen on the mailing lists where professional mail admins gather.
You'll find readily-verified postings from people talking about their
scaling issues in very large organisations including ISPs and
multinational companies. See for yourself (and if you want to do a
survey based on that sort of information I'd might be able to help.)
↑ As of February 2007 there is evidence that the Fedora 7 test1 default MTA is Exim, and debate as to whether that is a good thing
↑ The Georgi Guninski vulnerabilities have been multiply confirmed, and one of them is a root vulnerability. DJB rejects this
in typical style claiming it is an improbable configuration, but
security analysis is always seeking improbable setups! DJB is wrong,
but qmail still has an outstanding record.
↑ Actually Postfix also has a file called master.cf which is both somewhat obscure and very rarely touched. However, it does exist!
↑ As Bruce Schneir has explained in numerous articles and demonstrations
↑ Which doesn't stop me learning from the others -- thankyou NetBSD for ISBN 0-201-79940-5 and ISBN 0-321-16607-8 !
↑ That is, well-justified security design considerations. To take one example, Postfix has removed support for the @
character appearing in email usernames, and Wietse has stated he will
never restore that ability. There are a lot of people who need this
support, and Postfix cannot practically provide this, by design.
Similarly there is much less flexibility for performing content
scanning by external programs partway through SMTP processing, so there
will never be an equivalent to Exim's Exiscan interface.

REFERENCE
http://servertune.com/kbase/entry/289/

Command Center

Tuesday, March 30, 2010

Exim, or Postfix, or Qmail, or Sendmail :: MTAs Comparison

Labels

Blog Archive

My Blog List