Tuesday, June 22, 2010

building a secure web server with centos-5

SkyHi @ Tuesday, June 22, 2010

Part 1 | Part 2 | Part 3 | Troubleshooting


This is an updated version of my original LAMP (Linux Apache MySQL and Perl/PHP) guide that was based on CentOS 4. Now updated and tweaked for CentOS 5, I will take you through the steps required to build a secure Linux web server (LAMP) on CentOS 5.

I have a background working for an ISP, so I’ve based this build on the same configuration many hosting providers use. It supports virtual hosts (multiple websites), secure FTP access, locked down SSH access, and a sensible directory structure.

If you follow this guide, you will get a web server up and running within a couple of hours depending on whether you follow it step by step, or prefer to experiment first. If you are new to Linux then give it a try and learn something new, you never know you may surprise yourself!

Good luck!

A word on web hosting

Before you get started with your server build, I’d like to talk about where you are going to host this server. There have been a number of developments in the past few years in regards to people hosting websites on their home ISP broadband connection.

Firstly, did you know that many of the major search engines will not crawl your website if it’s hosted on a residential IP address? What I mean by residential is an IP address provided by the likes of an ADSL or broadband provider (e.g. Virgin Media, BT, PlusNet, here in the UK). It will depend on the ISP whether you get a ‘fixed’ IP address or not, and even if you do your site will simply be ignored by the major search engines as they will recognise the IP address in a residential (non ISP) range. Don’t get me wrong, running web servers at home is great fun but I would recommend you avoid it unless it is purely for testing.

So, where is the best place to host your shiny new CentOS web server?

I would highly recommend getting a VPS (Virtual Private Server). There are plenty of hosting companies offering Virtual Private Servers, and VPS hosting is getting cheaper.

I would personally recommend Linode, I have been using them since 2006 and recently they have started hosting virtual servers in the London, UK as well as the USA. They provide a generous amount of bandwidth, starting at around 200GiB per month with 16GB+ of disk space depending on which option you go for. You can choose from a massive list of Linux distro’s, including CentOS 5! They use Xen virtualisation and they have an excellent web interface for accessing your server stats, server controls, DNS, and console access.

Getting Started

CentOS 5 is completely free and developed by a team of core developers at a North American Enterprise Linux vendor. In turn the core developers are supported by an active user community including system administrators, network administrators, enterprise users, managers, core Linux contributors and Linux enthusiasts from around the world.

CentOS has numerous advantages over some of the other clone projects including: an active and growing user community, quickly rebuilt, tested, and QA’ed errata packages, an extensive mirror network, developers who are contactable and responsive, multiple free support avenues including IRC Chat, Mailing Lists, Forums, a dynamic FAQ. Commercial support is offered via a number of vendors.

CentOS 5 is distributed on six CD’s, all of which are available for download from the CentOS website.

Download phpMyAdmin

This is optional, but I would highly recommend this excellent web interface for administering your MySQL databases. I have used this in the past to provide customers with their own phpMyAdmin login username, so they can manage their databases easily.


Installing CentOS

NOTE: If you are using a virtual private server (VPS) provided by a hosting company such as Linode, and CentOS is already installed then skip ahead to the next section.

  • Insert CD of the CentOS 5 installation CD and boot your server.
  • At the installation menu, just press ENTER for the graphical installation wizard.
  • When prompted for an installation type select custom installation.
  • Use automatic partitioning for the disks.
  • Remove all partitions from system (make sure you are happy to wipe all existing data!!).
  • Boot Loader: Leave default settings.
  • Network Configuration: Configure this with an internal IP address and DNS name.
  • Firewall: Select ‘No firewall’ as this will be installed and configured later.
  • SELinux: Set to ‘Disabled’. This is still very experimental so I would leave this switched off unless you really know what you are doing.
  • Authentication: Set a secure root password using random characters and numbers (upper an lower case).
  • Package Selection: Choose minimal configuration (Other packages can be installed at a later stage according to the server role).

WARNING!! – The server should not be connected to the internet until the configuration is completed and secure!

Updating the System

Now that you have CentOS 5 installed, we need to make sure it’s up to date and then do some basic security configuration with SSH. Unlike CentOS 4, you no longer have to import the RPM key to update and install software. It does this for you.

To check for updates type the following:

# yum check-update

Now perform the update process. Note, the -y is to accept all updates which I recommend as it’s a clean installation.

# yum –y update

Setting the clock

I strongly advise that you setup the timezone and clock correctly. First, you set /etc/localtime to link to the correct timezone, then either set the time manually or configure NTPD to syncronise with an internet time server such as pool.ntp.org.

Example of setting the timezone to GMT:

# ln -sf /usr/share/zoneinfo/GMT /etc/localtime

Setting the hardware clock:

# vi /etc/sysconfig/clock

System Services Configuration

As this is going to be a finely tuned web server, we don’t want uneccessary daemons running! Firstly, lets list all the daemons that have been configured to run at startup.

# chkconfig –list|grep on

You should now get an output similar to the following:

anacron 0:off 1:off 2:off 3:off 4:off 5:off 6:off

atd 0:off 1:off 2:off 3:on 4:on 5:on 6:off

crond 0:off 1:on 2:off 3:on 4:on 5:on 6:off

cups 0:off 1:on 2:off 3:on 4:on 5:on 6:off

haldaemon 0:off 1:on 2:off 3:off 4:on 5:on 6:off

messagebus0:off 1:on 2:off 3:off 4:on 5:on 6:off

network 0:off 1:on 2:off 3:off 4:on 5:on 6:off

syslog 0:off 1:on 2:off 3:off 4:on 5:on 6:off

Your list will probably be a lot longer as this is just an example, but what you can see here is the different run levels and their on/off status. Most daemons start at run level 3. Now lets switch off the daemons that aren’t needed. I’ve listed a few more here that you are likely to find.

# chkconfig cups off

# chkconfig apmd off

# chkconfig netfs off

# chkconfig pcmcia off

# chkconfig smartd off

# chkconfig anacron off

# chkconfig mdmonitor off

# chkconfig isdn off

NOTE: If you are using Linode then you should also switch off Kudzu (hardware detection) as this serves no purpose on a virtual UML system.

Host Access (TCP_WRAPPERS)

There are two host access files (/etc/hosts.allow and /etc/hosts.deny), that are part of the TCP_WRAPPER package. This makes it possible to allow or deny access to certain services based on the IP.

Edit the hosts.allow and hosts.deny files:

# vi /etc/hosts.allow




# vi /etc/hosts.deny


The <IP ADDRESS> above is the internet IP you are connecting from (don’t include < or >). You can enter multiple IP address here (separated by spaces) or to allow SSH from any IP just replace with ALL.

The root account should never be able to login via SSH (without first logging in as a user). You must change this, so edit /etc/ssh/sshd_config and ensure the following is set:

# vi /etc/ssh/sshd_config

Change the following lines as follows:

PermitRootLogin no

Protocol 2

Note: Some of these lines may already exist but will be commented out using #. To enable these commands the # needs to be removed.

Add Default Accounts

Before proceeding with any of the steps below, first create a user account that you will use to log in to this server. This account will be used for SSH connections.

# adduser <username>

# passwd <username>

You should now have access to the server via SSH. Download PuTTY and make sure it works.

Part 2: Configuring the Server

Configuring CentOS

  • Edit /etc/hosts and /etc/sysconfig/network with hostnames
  • Install Packages

Example of /etc/hosts:

# Do not remove the following line, or various programs

# that require network functionality will fail. localhost www.mydomain.com

Example of /etc/sysconfig/network



Installing Packages

# yum -y install httpd openssl-devel openssl mod_ssl vsftpd rpm-build rpm-devel autoconf automake lynx gcc

# yum -y install mysql mod_auth_mysql mysql-devel mysql-server

# yum -y install mod_python python python-devel

# yum -y install perl mod_perl mod_perl-devel openssl-perl perl-Convert-ASN1 perl-Date-Calc perl-DateManip perl-HTML-Parser perl-libwww-perl perl-CPAN perl-DBD-MySQL perl-XML-Parser

# yum -y install php-devel php php-domxml php-gd php-mbstring php-mysql php-ncurses php-pear

# yum -y install webalizer

# yum -y install sendmail sendmail-cf

Creating Directory Structure

All websites will be held in /home/.sites/. The first site that needs to be created is the _default site, which will be used as this servers default website.

# mkdir /home/.sites

# cd /home/.sites

# mkdir _default

# cd _default

# mkdir logs private cgi-bin web

# cd web

# mkdir stats

Now change the ownership of these directories to adminftp as follows:

# cd /home/.sites

# chown adminftp _default -R

Configure the required system services sto start at boot:

# chkconfig httpd on

# chkconfig mysqld on

# chkconfig vsftpd on

# chkconfig sshd on

Configuring Apache

Apache runs as the httpd service, and it’s configuration file is contained in /etc/httpd/conf. To run in a ‘virtual’ hosting environment, we will now configure the httpd.conf file. But first things first, lets backup the httpd.conf file!

# cd /etc/httpd/conf

# cp httpd.conf httpd.conf.backup

Now edit httpd.conf and make the following changes, substituting mydomain.com for your own domain:

ServerAdmin admin@mydomain.com

ServerName www.mydomain.com:80

NameVirtualHost *:80

DirectoryIndex index.html index.htm index.html.var

ServerAdmin admin@mydomain.com

DocumentRoot /home/.sites/_default/web

ServerName www.mydomain.com

ScriptAlias /cgi-bin/ /home/.sites/_default/cgi-bin/

ErrorLog /home/.sites/_default/logs/error_log

TransferLog /home/.sites/_default/logs/access_log

Options FollowSymLinks

Options +Includes

AllowOverride All

Save and exit the httpd.conf file and then restart the httpd service.

# /etc/init.d/httpd restart

Configuring VSFTP

VSFTP stands for Very Secure File Transfer Protocol. However using the installation defaults isn’t actually that secure as it allows anonymous access and doesn’t restrict which users can access the servers FTP service. In order to harden the security of VSFTP, several configuration changes must be made. Please note that some of these lines may be commented out by default, with a #, so remember to remove this if required.

# vi /etc/vsftpd/vsftpd.conf






ftpd_banner= **** WARNING - Your actions are being logged ****







Next, we need to configure vsftpd.userlist and specify which users can FTP to the server. This compliments the userlist_deny setting in vsftpd.conf. When set to NO, this makes the vsftpd.userlist file a list of users that ARE allowed to log in.

# vi /etc/vsftpd/user_list

Remove all of the users that are listed in this file by default, and add ONLY the users that require access to the FTP server.

Configuring MySQL

MySQL is configured using /etc/my.cnf, but prior to any configuration this configuration file has only the bare minimum required to start the MySQL daemon. However there are 5 templates that we can base the configuration on; my-huge.cnf, my-innodb-heavy-4G.cnf, my-large.cnf, my-medium.cnf, and my-small.cnf. These are each configured depending on the RAM and the priority that MySQL has on this server.

We will use my-medium.cnf as this has been based on a web server where MySQL is not the primary role of the server but could have frequent use. If you are hosting a few sites with light use then use my-small.cnf.

# cd /usr/share/doc/mysql-server-

# cp my-medium.cnf /etc/my.cnf

Once this has overwritten my.cnf, the mysqld service must be restarted.

# /etc/init.d/mysqld restart

Now the root password for MySQL must be set using the following command. Do NOT use the same root password as the Linux root password.

# mysqladmin -u root password

Setting up phpMyAdmin

Administration of MySQL is carried out using phpMyAdmin which is a free open source software package licensed under the GNU. Nearly all hosting providers use this, and is pretty much the de-facto standard these days.

1) Download phpMyAdmin,

2) Extract the contents to a directory called phpMyAdmin (case sensitive)

3) Transfer the phpMyAdmin directory to /home/.sites/_default/web (Use FTP and login using the user you setup previously, as described in the Configuring FTP using VSFTPD section).

Now we need to configure the confic.inc.php file as follows:

# vi /home/.sites/_default/web/phpMyAdmin/config.inc.php

Look for:

$cfg['Servers'][$i]['auth_type'] = 'config’;

Change ‘config’ to ‘http’ then restart MySQL.

# /etc/init.d/mysqld restart

You should now be able to log into phpMyAdmin by going to http://www.mydomain.com/phpMyAdmin. Log on using ‘root’ and the password you specified for MySQL. You will now be presented with the phpMyAdmin web control panel. By default, MySQL contains a test database which is not required, so as a security measure, this needs to be deleted. To do this, click on ‘Databases’, then tick test. Click ‘Drop’ and then click ‘Yes’ to confirm.

Part 3: Adding websites

In the following steps, we will base our configuration on a fictitious company called Happy Burger who has already registered the domain name, happyburger.net. We will point www.happyburger.net to the IP address of this web server. When you are creating your own site, substitute the customer name and domain name for that of the actual customer. * If a Happy Burger really exists, then this is in no way associated with them and is purely coincidental.

Create a User Account

The first step is to create a user account that will be associated with this website and be used to authenticate via FTP. When creating the password, make sure that it is at least 8 characters, alphanumeric, mixed case and includes numbers. I usually generate random passwords for this.

# adduser –s /sbin/nologin happyburger

# passwd happyburger

Creating the directory structure

Each website must have the following directory structure in order to support access logs, web statistics, .htpasswd files, CGI scripts and the public web directory.

/home/.sites/happyburger/: This path will contain a directory for each website. Each directory should be named after the customer name, in lowercase.

/home/.sites/happyburger/web/: This path contains the website contents (public root).

/home/.sites/happyburger/web/stats/: This path will contain the Webalizer statistics, and is password protected using .htaccess.

/home/.sites/happyburger/private/: This path is not accessible from the internet, and contains the .htpasswd file.

/home/.sites/happyburger/cgi-bin/: Apache uses this path as the CGI script directory, by using a script alias.

/home/.sites/happyburger/logs/: This path stores the log files that Apache generates.

Create the structure as follows:

# cd /home/.sites

# mkdir happyburger

# cd happyburger

# mkdir web cgi-bin private logs

# cd web

# mkdir stats

Now change the ownership of these directories, substituting <username> (in this case, happyburger):

# cd /home/.sites

# chown <username> happyburger -R

Configuring Apache

As Apache will be configured using multiple ‘virtual hosts’ we need to create a separate configuration file for each virtual host. To do this we will create a vhost directory, and configure the Apache configuration file to read each of these virtual host configurations.

# cd /etc/httpd/vhost (If this directory does not exist then you will need to create it)

Now we will create the virtual host configuration file for this particular website.

# vi happyburger.conf

Now enter the following into the newly created configuration file:

<VirtualHost *:80>

ServerAdmin admin@happyburger.net

DocumentRoot /home/.sites/happyburger/web

ServerName www.happyburger.net

ServerAlias happyburger.net

ServerAlias www.happyburger.com

ServerAlias happyburger.com

ScriptAlias /cgi-bin/ /home/.sites/happyburger/cgi-bin/

<Directory /home/.sites/happyburger/web>

Options FollowSymLinks

Options +Includes +ExecCGI

AllowOverride All



Once this has been saved, we will then need to configure Apache to include this in the main configuration.

# vi /etc/httpd/conf/httpd.conf

At the end of the configuration file add the following line:

Include /etc/httpd/vhost/happyburger.conf

Now restart httpd:

# /etc/init.d/httpd restart

Configuring FTP (VSFTP)

Before the new account can login with FTP, you must add the new user to vsftp.user_list which contains a list of all accounts permitted to use the FTP service.

# vi /etc/vsftpd.user_list

Add the new user to the list.

Problems with Sendmail

When setting up my first Linux web server it was actually sendmail that caused me the most trouble. If you get the following errors:

<< 550-Verification failed for
<<< 550-Unrouteable address
<<< 550- does not appear to be valid. Sender verify
<<< 550 failed.
550 5.1.1 ... User unknown

1) This is likely to be a problem with the hostname of the server. Check to make sure the hostname of the server (e.g. linuxweb.mydomain.com) can actually be resolved over the internet. The hostname must be the same in /etc/hosts, /etc/sysconfig/network, and /etc/mail/relay-domains.

2) The /etc/mail/local-host-names and /etc/mail/relay-domains file should contain the hostname of the server, and also localhost. If the relay-domains file does not exist, then you can create it.

3) Check /etc/hosts.allow and make sure it has sendmail:ALL

4) This is unlikely but it may be a problem with the sendmail configuration (/etc/mail/sendmail.cf). Do not edit /etc/mail/sendmail.cf directly, instead edit /etc/mail/sendmail.mc which is a meta-config file. When done, you can use m4 sendmail.mc > sendmail.cf to write the meta-config to the sendmail.cf file.

The sendmail.mc file has various entries beginning with dnl (do not load). For sendmail to work properly on a virtual hosting server then make sure you add dnl to the following:

dnl FEATURE(delay_checks)dnl
dnl FEATURE(`mailertable',`hash -o /etc/mail/mailertable.db')dnl
dnl FEATURE(always_add_domain)dnl
dnl FEATURE(`blacklist_recipients')dnl
dnl FEATURE(`relay_based_on_MX')dnl
dnl FEATURE(masquerade_envelope)dnl
dnl FEATURE(masquerade_entire_domain)dnl

Then, the following entries should be enabled (without dnl at the beginning)

FEATURE(`virtusertable',`hash -o /etc/mail/virtusertable.db')dnl
FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl
FEATURE(`access_db',`hash -T -o /etc/mail/access.db')dnl

5) As a final resort, re-install sendmail then go back to step 1.

# yum remove sendmail sendmail-cf
# yum install sendmail sendmail-cf