This page describes how to compile and install sendmail with email aliases and procmail support. Email aliases let you create mailing lists and give alternate names or addresses for users. Procmail is a tool for filtering email to remove spam, sort mail, etc. The procedure for configuring procmail is described 
here and 
here. 
If you are running a version sendmail earlier than 8.13.6, you need to upgrade because of a serious security problem. As of  version 8.12, you must also create a new user smmsp and change  the permissions of various files as described in the file  sendmail/SECURITY. 
 Preparation 
-  Back up the following files:      
| /etc/mail/sendmail.cf
/etc/sendmail.cf
/etc/mail/submit.cf
/usr/sbin/sendmail |  
 
-  Create a user and group called `smmsp':       
| useradd smmsp
groupadd smmsp |  
 This step is essential, as sendmail will not start up unless      it has its own userid.
 The entries in /etc/passwd and /etc/group should look something      like this:
 
| cat /etc/passwd | grep smmsp
smmsp:x:25:25:Sendmail:/home/smmsp:/bin/false
cat /etc/group | grep ^smmsp
smmsp:x:25:smmsp |  
 
 Install libdb 
If you don't care about email aliases, skip this step. Otherwise, if you don't have libdb on your system ('locate libdb'), get it       from www.sleepycat.com and install it. For Unix/Posix systems:       
| cd db-4.2.52/build_unix
../dist/configure
make
su
make install | 
Make a note of where the libraries were installed      (e.g., /usr/local/BerkeleyDB.4.2/lib).      
After installing, it may be necessary to perform these three extra       steps (as root), depending on whether an old version of libdb is already present.      
-  First, copy db.h to /usr/include. This is essential, as the db.h            must match the library. If it doesn't, sendmail won't compile.           
| cd db-4.2.52
cp build_unix/db.h /usr/include/  |  
 
 
-  Second, get rid of the old libdb libraries.           
| cd /usr/lib
mkdir libdb-old
mv libdb* libdb-old |  
 
 
-  Third, add the new libdb path (/usr/local/BerkeleyDB.4.2/lib)           to /etc/ld.so.conf and  reconfigure the run-time linker.           
| vi /etc/ld.so.conf
ldconfig  |  
 
 Compile and install sendmail 
Compile sendmail. In some linux distributions, you will need to      have the source code for a kernel installed before sendmail will      compile. However, sometimes this can be avoided by creating an      empty 
 version.h  file.         
| touch /usr/include/linux/version.h
tar -xzvf sendmail.8.11.6.tar.gz
cd send* | 
-  First, edit the config files if you plan to      use procmail.      
| cd cf/cf
cp generic-linux.mc sendmail.mc |  
 Edit   sendmail.mc       and add at the end. Add any other commands as needed.
-  Create and install new config files using the following command:      
| sh Build sendmail.cf  
sh Build submit.cf  
su
make install-cf |  
 In earlier versions, the submit.cf file was not copied automatically       if an old  version of the file already existed.      If you try to use an old version of submit.cf, when you start sendmail       it will say:
| NOQUEUE: SYSERR(root): can not chdir(/var/spool/clientmqueue/):
Permission denied |  
 This error can also occur if sendmail is running as the wrong      user or group.
 
 Alternatively, install the config files manually.
| cd cf/cf
cp config.cf /etc/mail/sendmail.cf
cp submit.cf /etc/mail/ |  
 
-  Create a devtools/Site/site.config.m4 file containing the paths       for libdb. Change the paths to indicate wherever the libdb       libraries and include files are located.        
| APPENDDEF(`confMAPDEF', `-DNEWDB')dnl
APPENDDEF(`confINCDIRS', `-I/usr/include/db2')
APPENDDEF(`confLIBDIRS', `-L/usr/lib')
APPENDDEF(`confLIBS', `-ldb')  |  
 The exact syntax will vary depending on your system. On one of      our computers, where we installed a new libdb, the file contained      the following four lines:
| APPENDDEF(`confLIBS', `-lssl -lcrypto -ldb')
APPENDDEF(`confLIBDIRS', `-L/usr/local/ssl/lib -L/usr/local/ssl/lib
          -L/usr/local/BerkeleyDB.4.2/lib -L/usr/lib')
APPENDDEF(`confINCDIRS', `-I/usr/local/ssl/include -I/usr/include')
APPENDDEF(`confMAPDEF', `-DNEWDB')dnl  |  
 Sometimes compilation fails with undefined references to       `pthread_mutex_trylock' and other pthread functions. If this       happens, change the confLIBS line to
| APPENDDEF(`confLIBS', `-ldb -lpthread')  |  
 
 NOTE: If you use the site.config.m4.example      file that is included with sendmail, be sure to comment out items you       don't want, or sendmail will probably not compile.
 
 
-  Compile sendmail      If you change any configuration files and need to compile a second time,       use the -c option.      The -c option rebuilds all the files to include your changes.      It should automatically add the correct options to the commands, for example:      
| cc -O2 -I. -I../../sendmail   -I../../include  \
     -I/usr/include/db2 -DNEWDB -DNEWDB -DNOT_SENDMAIL     \
     -c -o vacation.o vacation.c  |  
 
 IMPORTANT: It's necessary to watch the screen continuously      while sendmail is building, because the build script does not      stop if there's an error.
 
 If your linux system is really old, you will get error messages like
| map.c:2084: `DB_FCNTL_LOCKING' undeclared (first use in this function) |  
 This means you need to install a newer copy of the Berkeley DB       libraries from www.sleepycat.com.
-  Check to make sure it created an executable sendmail in the directory    obj.Linux.#.#.##.i686/sendmail. If it bombs out, edit the file     include/libsmdb/smdb.h,      removing the lines      
| # ifndef NDBM
# ifndef NEWDB
ERROR   NDBM or NEWDB must be defined.
#  endif /* ! NEWDB */
# endif /* ! NDBM */  |  
 and type    sh Build -c  again.
-  Install the new sendmail      Scroll back through the output and make sure there were no errors.       If it says:      
| install: invalid group `smmsp'
make[1]: *** [install-sendmail] Error 1 |  
 you must fix the groups and repeat.
-  Set the correct permissions and ownerships:      
| mkdir /var/spool/clientmqueue
chown smmsp:smmsp /var/spool/clientmqueue
chmod 770 /var/spool/clientmqueue
chmod 700 /var/spool/mqueue
chown root.root /var/spool/mqueue
chmod 777       /var/spool/mail
chown root.root /var/spool/mail
chmod 4555       /usr/sbin/sendmail
chown root.smmsp /usr/sbin/sendmail |  
 NOTE:  /var/spool/mail  must be world       read/write, otherwise clients can't open their inbox.      The permissions for individual mailboxes for each user should be 600:
| -rw-------    1 daboss users     20219537754 2006-03-31 10:15 daboss |  
 
 Check to make sure of the following:
| -r-sr-xr-x   root    smmsp  /usr/sbin/sendmail
drwxrwx---   smmsp   smmsp  /var/spool/clientmqueue
drwx------   root    root   /var/spool/mqueue
-r--r--r--   root    bin    /etc/mail/sendmail.cf
-r--r--r--   root    bin    /etc/mail/submit.cf |  
 
 
-  Add a local-host-names file.      
| cp /etc/sendmail.cw /etc/mail/local-host-names |  
 If local-host-names doesn't exist, create one. It should      contain a single line:      If the server has more than one name, add them to this file, one       per line.
 Testing sendmail 
-  First, as root, test it to make sure the 'newaliases' command works.      If it says      
| Cannot rebuild aliases: no database format defined
Cannot create database for alias file /etc/mail/aliases  |  
 It means sendmail couldn't find the libdb files. Check that       db.h is in /usr/include, that the new libraries are in their      correct location, the old ones have been removed, and that      /etc/ld.so.conf has a line indicating the path to the new      libraries.
-  Restart inetd to make sure imap and pop3 aren't crashing. 
-  Make sure sendmail is running. Unlike earlier versions, it sometimes now       has two or more entries:      
| ps -aux | grep mail
root   Mar30   0:01 sendmail: accepting connections                 
smmsp  Mar30   0:00 sendmail: Queue control                         
smmsp  Mar30   0:01 sendmail: running queue: /var/spool/clientmqueue |  
 
-   Test to make sure users can send and receive mail in all of the following:      - smtp client (e.g. pine)      
- http (e.g. squirrelmail via browser)      
- pop3 client (e.g. windows)      
- imap4 client (e.g. windows)      
 Also, check the system logs to make sure there are no weird error messages.
-  If you like pain and you want to change sendmail.cf, the easiest way is to      edit  sendmail.mc  and then create it using       
| m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf |  
 
-  You may wish to test to ensure that your sendmail installation is not       acting as an open mail relay by typing the command:        telnet relay-test.mail-abuse.org       There is also a web service at       www.abuse.net/relay.html. You can download software to test it yourself      from http://www.monkeys.com/mrt/.      To be valid, the testing must be done using an account on a machine in some       other domain (such as a temporary email account on abuse.net).       
 
 However, testing this way is risky. If the testing software discovers       that you're an open relay, your hostname or even your entire domain       may end up in their public blacklist, thereby blocking valid email sent       by your users. Last time I checked, the testing site www.ordb.org does this.       We have found these blacklists are so inaccurate that we had to disable      the blacklist check in spamassassin to avoid      losing incoming mail. Thus, remote testing may do more harm than good.
Error messages
 Problem: SMTP greeting failure: 421 SMTP connection went away!  
Solution: Ownership or permissions of sendmail binary are wrong. Set it to 4555. Or, sendmail is not running. 
 Problem: pthreads errors  when building sendmail 
| /usr/local/BerkeleyDB.4.2/lib/libdb.so: undefined reference to `pthread_condattr_setpshared'
/usr/local/BerkeleyDB.4.2/lib/libdb.so: undefined reference to `pthread_mutexattr_destroy'
/usr/local/BerkeleyDB.4.2/lib/libdb.so: undefined reference to `pthread_mutexattr_setpshared'
/usr/local/BerkeleyDB.4.2/lib/libdb.so: undefined reference to `pthread_mutexattr_init'
/usr/local/BerkeleyDB.4.2/lib/libdb.so: undefined reference to `pthread_mutex_trylock' | 
Solution: Turn off pthreads in sendmail by changing the confLIBS line in the file devtools/Site/site.config.m4 to: 
| APPENDDEF(`confLIBS', `-ldb -lpthread') | 
and rebuild with sh Build -c.  
 Problem: Sendmail won't start 
| Initializing SMTP port (sendmail)/etc/mail/submit.cf: 
    line 430: readcf: option RunAsUser: unknown user smmsp: 
    No such file or directory  | 
Solution: Create user smmsp and group smmsp and rebuild sendmail.    
 newaliases  says: 
| su
Cannot rebuild aliases: no database format defined
Cannot create datase for alias file /etc/mail/aliases   | 
Solution:  This means sendmail couldn't find libdb. Compile and install libdb as described above, and rebuild sendmail.   
Unable to receive mail   Solution:  If using Suse Linux (or possibly other distributions), check /etc/sysconfig/mail and make sure it contains the line  
| SMTPD_LISTEN_REMOTE="yes" | 
See also 
 Pine email problems.   
 Tweaking sendmail configuration
-   Changing hostname:  Some computers are on two different      domains. To get sendmail to use the second domain in its headers      instead of your FQDN, add the following to submit.mc and       sendmail.mc: 
| FEATURE(always_add_domain)dnl
MASQUERADE_AS(got-syphilis.seconddomain.org)dnl
FEATURE(masquerade_envelope)dnl |  
 It may also be necessary to change the settings in your email      client. For example, in pine, if you have the line:
| customized-hdrs = Reply-To: nbonaparte@grand-armee.fr |  
 change it to
| Reply-To: nbonaparte@got-syphilis.seconddomain.org |  
 
-   Slowing down dictionary attacks and distributed denial  of service (DDoS) attacks:  (this information is from    http://www.technoids.org/dossed.html).
 Add the following to sendmail.mc:
| # block dos attacks
# no of seconds  connection rate
FEATURE(`access_db')dnl
FEATURE(`ratecontrol', ,`terminate')dnl
define(`confCONNECTION_RATE_WINDOW_SIZE',`10')dnl
# no of connections simultaneously
define(`confCONNECTION_RATE_THROTTLE', `8')dnl
FEATURE(`conncontrol', ,`terminate')dnl
define(`confBAD_RCPT_THROTTLE',`10')dnl
FEATURE(`greet_pause', `5000')dnl
# stop dictionary attacks
define(`confMAX_RCPTS_PER_MESSAGE', `10')dnl |  
 
-  Stopping server from sending receipts:  Some people try to check up on you by requesting a receipt when the email is delivered. Spammers may also use this to validate your users' email addresses. 
| # no receipts on delivery
define(`confPRIVACY_FLAGS', `noreceipts')dnl |  
 Check your email client; some clients send receipts as well.
After changing the .mc files, run sh Build sendmail.cf, sh Build submit.cf, su, make install-cf as before and restart sendmail. 
Reference: 
http://brneurosci.org/linuxsetup50.html