Thursday, December 10, 2009

Installing sendmail with aliases and procmail support

SkyHi @ Thursday, December 10, 2009
This page describes how to compile and install sendmail with email aliases and procmail support. Email aliases let you create mailing lists and give alternate names or addresses for users. Procmail is a tool for filtering email to remove spam, sort mail, etc. The procedure for configuring procmail is described here and here.
If you are running a version sendmail earlier than 8.13.6, you need to upgrade because of a serious security problem. As of version 8.12, you must also create a new user smmsp and change the permissions of various files as described in the file sendmail/SECURITY.


  1. Back up the following files:
  2. Create a user and group called `smmsp':
    useradd smmsp
    groupadd smmsp
    This step is essential, as sendmail will not start up unless it has its own userid.
    The entries in /etc/passwd and /etc/group should look something like this:
    cat /etc/passwd | grep smmsp
    cat /etc/group | grep ^smmsp

Install libdb

If you don't care about email aliases, skip this step. Otherwise, if you don't have libdb on your system ('locate libdb'), get it from and install it. For Unix/Posix systems:
cd db-4.2.52/build_unix
make install
Make a note of where the libraries were installed (e.g., /usr/local/BerkeleyDB.4.2/lib).

After installing, it may be necessary to perform these three extra steps (as root), depending on whether an old version of libdb is already present.
  1. First, copy db.h to /usr/include. This is essential, as the db.h must match the library. If it doesn't, sendmail won't compile.
    cd db-4.2.52
    cp build_unix/db.h /usr/include/ 

  2. Second, get rid of the old libdb libraries.
    cd /usr/lib
    mkdir libdb-old
    mv libdb* libdb-old

  3. Third, add the new libdb path (/usr/local/BerkeleyDB.4.2/lib) to /etc/ and reconfigure the run-time linker.
    vi /etc/

Compile and install sendmail

Compile sendmail. In some linux distributions, you will need to have the source code for a kernel installed before sendmail will compile. However, sometimes this can be avoided by creating an empty version.h file.
touch /usr/include/linux/version.h
tar -xzvf sendmail.8.11.6.tar.gz
cd send*
  1. First, edit the config files if you plan to use procmail.
    cd cf/cf
    Edit and add
    at the end. Add any other commands as needed.
  2. Create and install new config files using the following command:
    sh Build  
    sh Build  
    make install-cf
    In earlier versions, the file was not copied automatically if an old version of the file already existed. If you try to use an old version of, when you start sendmail it will say:
    NOQUEUE: SYSERR(root): can not chdir(/var/spool/clientmqueue/):
    Permission denied
    This error can also occur if sendmail is running as the wrong user or group.

    Alternatively, install the config files manually.
    cd cf/cf
    cp /etc/mail/
    cp /etc/mail/
  3. Create a devtools/Site/site.config.m4 file containing the paths for libdb. Change the paths to indicate wherever the libdb libraries and include files are located.
    APPENDDEF(`confINCDIRS', `-I/usr/include/db2')
    APPENDDEF(`confLIBDIRS', `-L/usr/lib')
    APPENDDEF(`confLIBS', `-ldb') 
    The exact syntax will vary depending on your system. On one of our computers, where we installed a new libdb, the file contained the following four lines:
    APPENDDEF(`confLIBS', `-lssl -lcrypto -ldb')
    APPENDDEF(`confLIBDIRS', `-L/usr/local/ssl/lib -L/usr/local/ssl/lib
              -L/usr/local/BerkeleyDB.4.2/lib -L/usr/lib')
    APPENDDEF(`confINCDIRS', `-I/usr/local/ssl/include -I/usr/include')
    APPENDDEF(`confMAPDEF', `-DNEWDB')dnl 
    Sometimes compilation fails with undefined references to `pthread_mutex_trylock' and other pthread functions. If this happens, change the confLIBS line to
    APPENDDEF(`confLIBS', `-ldb -lpthread') 

    NOTE: If you use the site.config.m4.example file that is included with sendmail, be sure to comment out items you don't want, or sendmail will probably not compile.

  4. Compile sendmail
    sh Build 
    If you change any configuration files and need to compile a second time, use the -c option.
    sh Build -c 
    The -c option rebuilds all the files to include your changes. It should automatically add the correct options to the commands, for example:
    cc -O2 -I. -I../../sendmail   -I../../include  \
         -I/usr/include/db2 -DNEWDB -DNEWDB -DNOT_SENDMAIL     \
         -c -o vacation.o vacation.c 

    IMPORTANT: It's necessary to watch the screen continuously while sendmail is building, because the build script does not stop if there's an error.

    If your linux system is really old, you will get error messages like
    map.c:2084: `DB_FCNTL_LOCKING' undeclared (first use in this function)
    This means you need to install a newer copy of the Berkeley DB libraries from
  5. Check to make sure it created an executable sendmail in the directory obj.Linux.#.#.##.i686/sendmail. If it bombs out, edit the file include/libsmdb/smdb.h, removing the lines
    # ifndef NDBM
    # ifndef NEWDB
    ERROR   NDBM or NEWDB must be defined.
    #  endif /* ! NEWDB */
    # endif /* ! NDBM */ 
    and type sh Build -c again.
  6. Install the new sendmail
    sh Build install     
    Scroll back through the output and make sure there were no errors. If it says:
    install: invalid group `smmsp'
    make[1]: *** [install-sendmail] Error 1
    you must fix the groups and repeat.
  7. Set the correct permissions and ownerships:
    mkdir /var/spool/clientmqueue
    chown smmsp:smmsp /var/spool/clientmqueue
    chmod 770 /var/spool/clientmqueue
    chmod 700 /var/spool/mqueue
    chown root.root /var/spool/mqueue
    chmod 777       /var/spool/mail
    chown root.root /var/spool/mail
    chmod 4555       /usr/sbin/sendmail
    chown root.smmsp /usr/sbin/sendmail
    NOTE: /var/spool/mail must be world read/write, otherwise clients can't open their inbox. The permissions for individual mailboxes for each user should be 600:
    -rw-------    1 daboss users     20219537754 2006-03-31 10:15 daboss

    Check to make sure of the following:
    -r-sr-xr-x   root    smmsp  /usr/sbin/sendmail
    drwxrwx---   smmsp   smmsp  /var/spool/clientmqueue
    drwx------   root    root   /var/spool/mqueue
    -r--r--r--   root    bin    /etc/mail/
    -r--r--r--   root    bin    /etc/mail/

  8. Add a local-host-names file.
    cp /etc/ /etc/mail/local-host-names
    If local-host-names doesn't exist, create one. It should contain a single line: 
    If the server has more than one name, add them to this file, one per line.

Testing sendmail

  1. First, as root, test it to make sure the 'newaliases' command works. If it says
    Cannot rebuild aliases: no database format defined
    Cannot create database for alias file /etc/mail/aliases 
    It means sendmail couldn't find the libdb files. Check that db.h is in /usr/include, that the new libraries are in their correct location, the old ones have been removed, and that /etc/ has a line indicating the path to the new libraries.
  2. Restart inetd to make sure imap and pop3 aren't crashing.
  3. Make sure sendmail is running. Unlike earlier versions, it sometimes now has two or more entries:
    ps -aux | grep mail
    root   Mar30   0:01 sendmail: accepting connections                 
    smmsp  Mar30   0:00 sendmail: Queue control                         
    smmsp  Mar30   0:01 sendmail: running queue: /var/spool/clientmqueue
  4. Test to make sure users can send and receive mail in all of the following:
    1. smtp client (e.g. pine)
    2. http (e.g. squirrelmail via browser)
    3. pop3 client (e.g. windows)
    4. imap4 client (e.g. windows)
    Also, check the system logs to make sure there are no weird error messages.
  5. If you like pain and you want to change, the easiest way is to edit and then create it using
    m4 /etc/mail/ > /etc/mail/
  6. You may wish to test to ensure that your sendmail installation is not acting as an open mail relay by typing the command: telnet There is also a web service at You can download software to test it yourself from To be valid, the testing must be done using an account on a machine in some other domain (such as a temporary email account on

    However, testing this way is risky. If the testing software discovers that you're an open relay, your hostname or even your entire domain may end up in their public blacklist, thereby blocking valid email sent by your users. Last time I checked, the testing site does this. We have found these blacklists are so inaccurate that we had to disable the blacklist check in spamassassin to avoid losing incoming mail. Thus, remote testing may do more harm than good.

Error messages

Problem: SMTP greeting failure: 421 SMTP connection went away!
Solution: Ownership or permissions of sendmail binary are wrong. Set it to 4555. Or, sendmail is not running.

Problem: pthreads errors when building sendmail
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_condattr_setpshared'
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_mutexattr_destroy'
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_mutexattr_setpshared'
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_mutexattr_init'
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_mutex_trylock'

Solution: Turn off pthreads in sendmail by changing the confLIBS line in the file devtools/Site/site.config.m4 to:
APPENDDEF(`confLIBS', `-ldb -lpthread')
and rebuild with sh Build -c.

Problem: Sendmail won't start
Initializing SMTP port (sendmail)/etc/mail/ 
    line 430: readcf: option RunAsUser: unknown user smmsp: 
    No such file or directory 

Solution: Create user smmsp and group smmsp and rebuild sendmail.

newaliases says:
Cannot rebuild aliases: no database format defined
Cannot create datase for alias file /etc/mail/aliases  

Solution: This means sendmail couldn't find libdb. Compile and install libdb as described above, and rebuild sendmail.

Unable to receive mail Solution: If using Suse Linux (or possibly other distributions), check /etc/sysconfig/mail and make sure it contains the line
See also Pine email problems.

Tweaking sendmail configuration

  1. Changing hostname: Some computers are on two different domains. To get sendmail to use the second domain in its headers instead of your FQDN, add the following to and
    It may also be necessary to change the settings in your email client. For example, in pine, if you have the line:
    customized-hdrs = Reply-To:
    change it to
  2. Slowing down dictionary attacks and distributed denial of service (DDoS) attacks: (this information is from
    Add the following to
    # block dos attacks
    # no of seconds  connection rate
    FEATURE(`ratecontrol', ,`terminate')dnl
    # no of connections simultaneously
    define(`confCONNECTION_RATE_THROTTLE', `8')dnl
    FEATURE(`conncontrol', ,`terminate')dnl
    FEATURE(`greet_pause', `5000')dnl
    # stop dictionary attacks
    define(`confMAX_RCPTS_PER_MESSAGE', `10')dnl
  3. Stopping server from sending receipts: Some people try to check up on you by requesting a receipt when the email is delivered. Spammers may also use this to validate your users' email addresses.
    # no receipts on delivery
    define(`confPRIVACY_FLAGS', `noreceipts')dnl
    Check your email client; some clients send receipts as well.
After changing the .mc files, run sh Build, sh Build, su, make install-cf as before and restart sendmail.