Saturday, June 18, 2011

A Django setup using Nginx and Gunicorn

SkyHi @ Saturday, June 18, 2011
This is a howto on setting up Django on a Linux (Ubuntu) system using Nginx as a reverse proxy and Gunicorn as a Django service.

Django, Gunicorn, Nginx

The conventional way to run Django in production these days is using Apache2 and mod_wsgi. While there’s nothing wrong with that approach, I prefer Nginx. I also like to be able to control Django server separately from the web server.

There are several production-ready servers for Django. The best seem to be Gunicorn and uWSGI, and Gunicorn seems the best supported and most active project.

When running Django server separately from the web server, we need a way to start, stop and restart the Django server. A popular way for doing it in Django world is Supervisor, altough, for Ubuntu users, Upstart might be less hassle.

You probably already have a Django project you want to deploy, but for completenes’ sake, the steps here will use an empty toy “Hello World” Django project:

First things first – you are using virtualenv, right? If not, you should.

virtualenv --no-site-packages test
cd test
source bin/activate
pip install gunicorn django createproject hello
cd hello
# to test the base setup works
python runserver


Testing Django with Gunicorn is as simple as:

gunicorn_django -b

For production, we might want a bit more options, and we want to make sure the server is executing in the correct environment. The easiest way is to create a shell script to set it all up:

set -e
# user/group to run as
cd /path/to/test/hello
source ../../bin/activate
exec /path/to/test/bin/gunicorn_django -w $NUM_WORKERS \
--user=$USER --group=$GROUP --log-level=debug \
--log-file=$LOGFILE 2>>$LOGFILE

The number of workers is number of worker processes that will serve requests. You can set it as low as 1 if you’re on a small VPS. A popular formula is 1 + 2 * number_of_cpus on the machine (the logic being, half of the processess will be waiting for I/O, such as database). YMMV.

Don’t forget to mark the script as executable (chmod ug+x You can run it from the command line for testing. Note that Gunicorn by default uses address (the same as Django debug server), which is fine if Nginx is on the same machine – you usually don’t want to have it wide open to anyone, and instead let Nginx handle incoming connections.

If you want to run several Django servers on the same machine, just make sure each uses a different port number.

Supervisor has extensive documentation, and this blog post is big already, so I’ll just point you to the official docs. The config file for running our server (/etc/supervisor/cont.d/hello.conf on Debian/Ubuntu) should look like this:

directory = /path/to/test/hello/
user = your_unix_user
command = /path/to/test/hello/
stdout_logfile = /path/to/logfile.log
stderr_logfile = /path/to/logfile.log

Test it with supervisortcl {start,status,stop} hello (as root).

Ubuntu alternative is Upstart, which has a similar config file (/etc/init/hello.conf). An example:

description "Test Django instance"
start on runlevel [2345]
stop on runlevel [06]
respawn limit 10 5
exec /path/to/test/hello/

Test it with service hello {start,status,stop} (as root).

If you don’t have it set up, you should also install Nginx. The install procedure varies from system to system. On Debian and Ubuntu systems, it’s as simple as apt-get install nginx, and other Linux distributions usually have equivalent commands.

Nginx is mostly a drop-in replacement for Apache for serving static files, though there are some things to set up if you need to run PHP code as well.

For our setup, we need Nginx to serve as the reverse proxy for the upstream server(s). To do so, we add a server section to the config file:

server {
listen 80;
# no security problem here, since / is alway passed to upstream
root /path/to/test/hello;
# serve directly - analogous for static/staticfiles
location /media/ {
# if asset versioning is used
if ($query_string) {
expires max;
location /admin/media/ {
# this changes depending on your python version
root /path/to/test/lib/python2.6/site-packages/django/contrib;
location / {
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 10;
proxy_read_timeout 10;
proxy_pass http://localhost:8000/;
# what to serve if upstream is not available or crashes
error_page 500 502 503 504 /media/50x.html;

Ubuntu and Debian systems keep Nginx config files in same layout as for Apache, so the above cold be added to /etc/nginx/sites-available/hello (and enabled by symlinking from sites-enabled directory). Use nginx -t for config test and nginx -s reload to reload the configuration.
That’s it

And that’s it. The services are really quite simple to set up once you know what goes where, the setup is flexible and performant, and the server environments are isolated so it’s possible to host many different services with varying requirements on the same machine.


Friday, June 17, 2011

Confusion in suPHP, suPHPexec and Apache suEXEC

SkyHi @ Friday, June 17, 2011
What is phpsuexec?

Phpsuexec is a deprecated feature in cPanel where php is setup as cgi instead of apache module. All shared hosting servers have been updated from phpsuexec to suPHP. This KB applies to such clients that are still using phpsuexec on their VPSs/Dedicated servers. Phpsuexec brings a new level of security to the way php is used.

1) php scripts execute using the permissions of userid of the account holder instead of user “nobody”

2) world writable folders (chmod 777) is not required for file uploads through php

3) The php file need to have ownership of the user to execute in the user account (by default it is).

4) php file (script) does not need 755 permissions. 644 is fine. In fact 400 or 600 is ok too (especially good for sensitive information).

5) php_flag or php_value can NOT be used in .htaccess files (It will result in Internal Server Error).

6) The php flags that do not work in .htaccess can be moved to php.ini file in the same folder where php script exists. However, the php flag/value will be in the format of php.ini and not that of .htaccess. For example this from .htaccess

php_flag register_globals off

will go into php.ini as:

register_globals off

If php.ini exists in the folder where the php scripts exists, it will take all values from it (and nothing will be taken from main php.ini.

7) If the folder that contains a php file/script is world writable (chmod 777), it will result in Internal Server Error. This is similar to cgi/perl scripts under suexec that do not like such permissions for security reason. The normal folder permissions should be 755.

8) Apache specific php functions do not work:

9) If your .htaccess file contains “Options” directive, it should have + or – with the directive to keep the ExecCGI active.

10) Symbolic links do not work for php scripts for security reasons.

11) Some web applications (OS commerce, ZenCart etc.) check if its configure.php file is writable (since php is being executed with userid it should be), so it will complain that its writable. Please change the permissions to 444 via ssh. (chmod 444 /path/to/configure.php)

12) HTTP authentication via php code does not work. However you can continue to use it via .htaccess or password protected folder feature of the control panel.

13) If you use “AddType application/x-httpd-php” in .htaccess, it should be set to “AddHandler application/x-httpd-php”

Similarly if you are using ForceType in .htaccess to force a file to be treated as php, you will need to change it to SetHandler.

What is suPHP?
What is suPHP? What is being changed from phpsuexec to suPHP?

suPHP is a tool for executing PHP scripts with the permissions of their owners. Currently our servers use phpsuexec which also executes PHP with the permission of their owners. However these are two different tools and there are some improvements with moving to suPHP.

Once suPHP is available on your server, you can login to your control panel and find a link ‘PHP Configuration’ under ‘Software/Services’. On that page

1. You can switch your account’s php to php4 or php5

2. You can read how to configure php and how suPHP works. It is similar to phpsuexec as explained above excluding some improvements mentioned below.

3. Download server wide php.ini for php4 or php5 and customize it for your own needs. You may need to do this regularly to keep your php settings in synch with server level settings e.g after Zend Optimizer upgrade.

Changes from phpsuexec to suPHP:

1. By default php CLI is php5. Here are the paths for your reference:

/usr/bin/php (php5 cgi)
/usr/local/bin/php (php5 cli)
/usr/php4/bin/php (php4 cgi)
/usr/local/php4/bin/php (php4 cli)

2. There are some significant improvments in suphp such as

* HTTP based authentication auth works via php
* symbolic links to php files also work
* permissions of public_html does not need to be changed for using shared SSL with php5
* Custom error pages will work with both php4 and php5

3. ionCube PHP Loader will be available server wide alongwith Zend Optimizer. If you use custom php.ini you will need to update it by downloading it from your control panel so that latest Zend Optimizer can load for your scripts as well.

4. If you are setting up custom php settings, the custom php.ini file will be required in a folder where the php script needs to execute. Or you can place php.ini anywhere and have this directive in public_html/.htaccess

suPHP_ConfigPath /home/username/php5-config

where username is your cpanel account username, and php5-config is just a folder name (you can name it anything) and it will pick php.ini from that folder. Yes, you can have php.ini outside of your webroot in suPHP. This is a new feature.

5. To activate php5 on a subfolder or in your whole account, this directive was added in .htaccess in phpsuexec

AddHandler application/x-httpd-php5 .php .php3 .phtml

or a variant of it. Now this must be proceeded by the marker comment to block cpanel from changing your settings:

# Use PHP5 as default
AddHandler application/x-httpd-php5 .php .php3 .phtml

Or if you use control panel to activate php5 (upgraded servers), then you do not need to manually add the above directive.

Other updates

1. For semi-dedicated clients, ffmpeg will be available via both php4 and php5.

2. mod_gzip is being installed as well on all servers.


One more time..

Differences between phpsuexec and regular php

When using the common PHP installation on a webserver, php runs as the user nobody and it doesn’t require the execute flag to be enabled.

The problem on this is that if mod_openbasedir is not installed, every user will be able to read your php files because everyone is virtually sharing the same username (nobody).

As most of you already know, PHP Files are not meant to be read, but parsed, and that is where the problem resides. PHP Files have to be parsed, otherwise everyone who is able to read your php file will see settings that you would probably want to keep private, such as your MySQL username and password.

PHPSUEXEC fixes all this because it requires php to be run as the file owner’s username. (for example: andre)

This is not everything it fixes though. PHPSUEXEC is also here to fix file ownership problems. This has been a common issue on a few Content Management Systems such as Joomla and also on the popular blog software: WordPress.

It also adds security to your files as you can use permissions such as 600 or 700 in your files and your visitors will still be able to view them (parsed) in their browsers.

PHPSUEXEC will also refuse to serve any pages that are at security risk, for example with 777 as permissions. (will generate an Internal Server Error)

PHP as an Apache Module

When PHP runs as an Apache module, PHP files work under the Apache user/group known as “nobody”. For example, when a PHP file needs to write to another file or create/remove a file, it does so under the name “nobody”. In order to allow “nobody” to do this, you need to set specific permissions on the file/directory, such as 777 – which translates to read/write/execute by user/group/world. This is insecure because you have not only allowed the webserver (Apache) to read/write to the file, you have also allowed everyone else on the server to read/write to the file as well!

Due to the above conditions, when a PHP file creates or uploads a new file under your account, the new file will be owned by the user “nobody”. If you FTP into your account, all files owned by “nobody” will not be available for you to move, rename or delete. In this case the only way to remove the “nobody” owned files would be through a file on the server or to contact support and ask for the file ownership to be changed back to your username.

PHP as a CGI with Suexec

When PHP runs as a CGI with Suexec, PHP files work under your user/group. PHP files no longer require loose permissions to function, now they will require strict permissions. Setting your directories or PHP files to 777 will cause them to produce a 500 Internal Server Error, this happens to protect your PHP files from being abused by outside sources.

Under PHPSuexec your directories and PHP files can have permissions no greater than 755 (read/write/execute by your username, read/execute by group/world). Since you own your files, your scripts can function in any directory your user has created and can’t be manipulated by any outside users, including “nobody”.

Now, when a PHP file creates or uploads a new file under your account, the new file will be owned by your username. You will no longer have to worry about the webserver taking over your files and even more important, you will no longer have to worry about a stranger reading or writing to your files either!

How To Enable WHM Apache PHP SuExec

By default PHP on WHM/Cpanel is loaded as DSO (Dynamic Shared Object) module and is run by the user “nobody” by default. Though this method of loading the PHP module is normally the fastest way to serve PHP request, running it as using user “nobody” will be a real pain in the ass if you are serving multiple sites run by multiple users, you will be for sure run into file permission problems.

This is where the SuExec comes in play, every executed PHP scripts will be executed by the user who owns the VirtualHost that is server the request, this method has a lot of drawbacks too on both speed and security.

Anyway, if you still want to enable it then read on below.

1. Login to your Web Host Manager as root account then under the Service Configuration menu, look for the “Configure PHP and SuExec” and click on it.

2. On the “Configure PHP and SuExec” page, under “alter configuration” section, look for the PHP handlers and then change its values to “cgi” and then set the Apache SuExec to On. (by default the value is on)

3. Finally, click on “Save new configuration” button and wait til the Apache server restarted and your done.

To verify that SuExec is working as intended, try to upload a file or create a folder using an upload file script on PHP.

That’s all about it.

Troubleshooting Internal Server Errors (Error 500)

Everytime an internal server error occurs, it will be added to your Error Log in cPanel. (cPanel »» Error Log). This will usually give you a clue on where the error resides. In most cases it will be either a permission error on a bad command in your .htaccess file (remember that all php values have to go to your php.ini file).

Directories that need to be written onto will no longer require 777 as permissions and phpsuexec will refuse to write or read on directories exposed with such permissions. You will have to chmod them to 755 always.

To simplify it, just remember that you should never have a file or folder with world-writeable permissions, because you no longer have to.


If you added a Mimetype to the system in order to run html files as php scripts (AddType as .htaccess command), you will have to remove it and add an ApacheHandler instead. This is easy to do though. Just log into your control panel, then click on Apache Handlers and add the following:

Extension: html (or htm) : AddHandler application/x-httpd-php


Disable suhosin for single domain

SkyHi @ Friday, June 17, 2011

Disable suhosin via php.ini

1. Login in server as root
2. copy main php.ini in users document root
cp /usr/local/lib/php.ini /home/username/public_html
3. Open php.ini using any editor
vi php.ini

; Misc Options
suhosin.simulation = On [On is for disable suhosin and off means enabled]
4. Save the file and exit

Disable suhosin using .htaccess
1. create .htaccess file under /home/username/public_html
and add this line
php_flag suhosin.simulation On
save and exit the editor.

If server is suphp enabled
enter following code in .htaccess
<files “.ht*">
deny from all</files>

suPHP_ConfigPath /home/username
Save the file and exit


How to protect/secure php.ini with SuPHP?

SkyHi @ Friday, June 17, 2011
When Apache is compiled as CGI/SuPHP, it allows users to create their own php.ini file under their home directory and modify the php values as per their wish. This may increase security concerns on the server and hence to protect/secure php.ini in SuPHP enabled servers, force every user to use a common php.ini file.

This can be achieved by defining the path of php.ini using suPHP_ConfigPath. To force users to use existing server side php.ini file, create suphp_configpath.conf

# pico /usr/local/apache/conf/userdata/suphp_configpath.conf

and add the following lines

#just specify the path to the php.ini file directory

suPHP_ConfigPath /usr/local/lib/

Once done, save the file and rebuild the Apache configuration so it picks up the changes.

# /usr/local/cpanel/bin/apache_conf_distiller –update –main
# /usr/local/cpanel/bin/build_apache_conf

To verify the include files, execute:


It will display the path to the .conf file you created. Restart the Apache service once

/scripts/restartsrv httpd

This will ensure all the users use the server side php configuration file. If you wish to keep the php.ini elsewhere, just change the value of “suPHP_ConfigPath” and follow the above steps.


Monday, June 13, 2011

Yes, You Can Run 18 Static Sites on a 64MB Link-1 VPS

SkyHi @ Monday, June 13, 2011
One thing I hated about WebHostingTalk is how much bad advice the so-called “professionals” are giving out to the world. Some poor college student asked in the VPS forums whether he is able to run 18 static HTML sites on Link-1 plan (64MB RAM, 2.5GB storage & 100GB/month data), and the typical responses are:
“I do not believe you can host 18 websites on 64MB of RAM. I’d bump that up to at least 128 or 256.” –nexbyte
“I really wouldn’t advise anything lower than 265MB RAM for website hosting.” –RikeMedia
(Well, there are some more optimistic comments but I mainly list out those “with things to sell”)
So, just trying to prove the point that yes, 64MB is more than enough to host 18 static sites, I decided to add a Link-1 Xen to my account and document the process. Btw, thanks to Dan @ VPSLink for getting my billing issue resolved :) You can get 10% recursive discount here, or 66% off for the first 3 months here.

Setting Up the VPS

After my order has been provisioned, I re-image the server with a Debian 5 “Lenny” image. I normally pick Debian or Ubuntu because apt-get uses much less memory than RedHat/Fedora’s equivalent, and it’s also my personal preference. I named my new VPS “endor” as I usually just name my boxes after Star Wars systems. Re-imaging a VPS is pretty fast — 2 minutes later I have my root password sent to my email address so I can ssh in to set up the new system.
$ ssh root@endor
root@endor's password:
Linux 66671 2.6.18-53.1.13.el5xen #1 SMP Tue Feb 12 14:04:18 EST 2008 i686

endor:~# free
             total       used       free     shared    buffers     cached
Mem:         65704      64008       1696          0       5616      44100
-/+ buffers/cache:      14292      51412
Swap:       131064          0     131064
endor:~# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Duo CPU     E4500  @ 2.20GHz
stepping        : 13
cpu MHz         : 2194.496
cache size      : 2048 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc up pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
bogomips        : 5558.81
Plenty of free memory and a single core of C2Duo E4500 —  although not a high-end Xeon CPU, but should be more than sufficient to  do what we need it to do. The next thing I want to do is to make sure  every package is up to date.

endor:~# apt-get update && apt-get upgrade
Get:1 lenny Release.gpg [386B]
Get:2 lenny Release [63.2kB]
Get:3 lenny/main Packages [5295kB]
Get:4 lenny/updates Release.gpg [197B]
Get:5 lenny/updates Release [40.8kB]
Get:6 lenny/contrib Packages [76.1kB]
Ign lenny/updates/main Packages/DiffIndex
Get:7 lenny/updates/contrib Packages [14B]
Get:8 lenny/updates/main Packages [50.6kB]
Fetched 5526kB in 4s (1330kB/s)
Reading package lists... Done

Setting Up Web Server

Okay. The 64MB VPS is now up and running. What should we do next? Installing a web server of course, so we can start serving our static pages! Which web server? Definitely not Apache as it would be a waste of valuable memory here. Again my personal favourite is Nginx (pronounces Engine X), which currently powers However, in this exercise I will go for Lighttpd because I found it easier to set up for abitary sites.
First of all — get Lighttpd installed.

endor:~# apt-get install lighttpd
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
Setting up libterm-readkey-perl (2.30-4) ...
Setting up libterm-readline-perl-perl (1.0302-1) ...
Setting up lighttpd (1.4.19-5) ...
Starting web server: lighttpd.
endor:~# ps -u www-data u
www-data  1690  0.0  1.5   5416  1008 ?        S    07:17   0:00 /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf
Plain vanilla stripped down and un-configured 32 bit Lighttpd sits around 1MB RSS — not bad.
Next, we need to get our websites up there and point Lighttpd to them. It’s a good idea to put the web sites in an organised structure inside the file system. I usually just place them this way:
  • /var/www//html
So if I have an HTML file at, it will sit on the file system at /var/www/ Unfortunately I do not have 18 static sites. For testing purpose I am only going to display a very basic HTML page at
endor:~# mkdir -p /var/www/
endor:~# echo '

Low End Box Rocks!

' > /var/www/
So now our “website” is ready — how does Lighttpd, our webserver, knows where to find the files corresponding to the website? That’s where Lighttpd’s mod_simple_vhost comes in handy.

endor:~# lighttpd-enable-mod simple-vhost
Available modules: auth cgi fastcgi proxy rrdtool simple-vhost ssi ssl status userdir
Already enabled modules:
Enabling simple-vhost: ok
Run /etc/init.d/lighttpd force-reload to enable changes
endor:~# /etc/init.d/lighttpd force-reload
Stopping web server: lighttpd.
Starting web server: lighttpd.
Now navigate to (which already has an A record to  my new VPS’s IP address) — here we have it! Low End Box Rocks!!!
You must be already familiar with DNS and know how to create records to point to IP addresses. For free DNS hosting I recommend EveryDNS, which has also been hosting LowEndBox’s domain.
You can now basically just dump static files at /var/www//html, with resolved to your VPS’s IP address, and you will have your static websites over there in no time. You do not even need to tell Lighttpd to reload, as mod_simple_vhost automatically maps the hostname to appropriate file name. Repeat it 18 times and problem solved!
At 1 single testing site with no traffic, Lighttpd sits at around 1.5MB RSS, although I doubt it would increase significantly when you increase the number of sites or the traffic. Lighttpd and Nginx are single-threaded poll-based asynchronised web servers so for static file serving, the bottle-neck would be disk/network IO rather than amount of memory or CPU performance.
There are still lots of memory left. Maybe we can have some fun.

Installing WordPress

So you think, “hey Low End Box rocks and it runs on WordPress. So maybe I will have that installed as well!” Wow. But WordPress is a content management system for creating dynamic websites! It simply cannot be possible on a 64MB VPS, the WHT crowd says! Grrr!! Let’s give it a try.
To run WordPress from your static-file serving Lighttpd, you need a few more packages — namely MySQL and PHP in CGI/FastCGI mode.

endor:~# apt-get install mysql-server php5-cgi php5-mysql
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
Creating config file /etc/php5/cgi/php.ini with new version
Setting up php5-mysql (5.2.6.dfsg.1-1+lenny2) ...
Setting up sgml-base (1.26) ...
Setting up xml-core (0.12) ...
Setting up mailx (1:20071201-3) ...
I know it installs whole lot of other junks but don’t worry — we will  live with them first and will try to optimise later. It also requires  you to set up the root password for MySQL server, and I conveniently  chose the most obscured password in this exercise — “root” (yes, don’t  use that because I am already using it as my root password :)
We then need to configure lighttpd to handle PHP files.

endor:~# cat > /etc/lighttpd/conf-enabled/10-cgi-php.conf
server.modules += ("mod_cgi")
cgi.assign = (".php" => "/usr/bin/php5-cgi")^D
endor:~# /etc/init.d/lighttpd force-reload
Stopping web server: lighttpd.
Starting web server: lighttpd.
Done! It should be able to serve PHP files. Just to test it out:
endor:~# echo '' > /var/www/
Now navigate to — you should be able to see the output of phpinfo() function. What we are going to do next is to set up a WordPress blog under already provides a great tutorial on installing WordPress, but let’s do it step by step on the command prompt.
My plan:
  • Create database “test_blog”
  • Download the latest WordPress and unzip to
  • Set up configuration file and run the WordPress install
  • Update Lighttpd to provide clean URL, aka Pretty Permalinks.
Let’s go!
endor:~# mysqladmin -uroot -proot create test_blog
endor:~# wget
Connecting to||:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: `latest.tar.gz'


2009-03-17 13:03:15 (1.01 MB/s) - `latest.tar.gz' saved [1624416]

endor:~# tar zxf latest.tar.gz -C /var/www/
endor:~# cd /var/www/
endor:/var/www/ mv wordpress blog
endor:/var/www/ mv blog/wp-config-sample.php blog/wp-config.php
endor:/var/www/ vi blog/wp-config.php 
When you are editing WordPress’ configuration file, set DB_NAME to “test_blog”, DB_USER and DB_PASSWORD  to “root” for something quick, dirty and potentially insecure. Here is  one final step — navigate to, and  WordPress will guide you through the rest of setup.
It is also relatively easy to set up pretty permalinks for WordPress on Lighttpd. In our example,

endor:~# cat > /etc/lighttpd/conf-enabled/lowendbox.conf
$HTTP["host"] == "" {
    $HTTP["url"] =~ "^/blog/" {
        server.error-handler-404 = "/blog/index.php"
endor:~# /etc/init.d/lighttpd force-reload
Stopping web server: lighttpd.
Starting web server: lighttpd.
That’s it! You can now go into WordPress to configure the desirable  Permalink Structure. Do note that the current WordPress dashboard page  is very resource intensive, as it fetches development blog, other WP news, incoming links, etc from various sources, concurrently  on separate PHP CGI processes. There might be plugins to turn off this  server-killing behavior (or just use older version of WordPress like  2.0.x which is still maintained). Likewise some WP caching plugin can be  very useful in reducing the load. Google them and you shall find.

Optimisation — Squeeze More Memory!

So now we have a Debian 5 web server box that can handle lots of static sites + a few WordPress blogs, and it fits “fine” on a 64MB Xen VPS. Let’s see what processes are running:

endor:~# ps aux
root       325  0.0  0.4   2032   292 ?        S<s  04:25   0:00 udevd --daem
root       879  0.0  0.4   2788   304 ?        Ss   Mar17   0:00 /bin/bash --
root      1216  0.0  0.0      0     0 ?        S    Mar17   0:00 [pdflush]
root      1649  0.0  0.2   3144   188 ?        Ss   Mar17   0:00 /usr/sbin/famd
root      6427  0.0  4.4   8024  2928 ?        Ss   Mar17   0:00 sshd: root@pts/
root      6429  0.0  2.3   2804  1564 pts/0    Ss   Mar17   0:00 -bash
root      6441  0.0  1.8  33092  1200 ?        Sl   Mar17   0:00 /usr/sbin/rsysl
root      6453  0.0  1.4   5284   976 ?        Ss   Mar17   0:00 /usr/sbin/sshd
root      6470  0.0  1.3   2048   896 ?        Ss   Mar17   0:00 /usr/sbin/cron
daemon    6482  0.0  0.8   1772   560 ?        Ss   Mar17   0:00 /sbin/portmap
www-data  6510  0.0  2.6   5848  1736 ?        S    Mar17   0:00 /usr/sbin/light
root      6572  0.0  1.7   2488  1156 pts/0    S    Mar17   0:00 /bin/sh /usr/bi
mysql     6611  0.0 26.2 143652 17228 pts/0    Sl   Mar17   0:00 /usr/sbin/mysql
root      6613  0.0  0.8   1636   536 pts/0    S    Mar17   0:00 logger -p daemo
103       6973  0.0  1.3   6112   908 ?        Ss   Mar17   0:00 /usr/sbin/exim4
root      6986  0.0  1.3   2308   908 pts/0    R+   00:01   0:00 ps aux
endor:~# free
             total       used       free     shared    buffers     cached
Mem:         65704      51424      14280          0        936      22588
-/+ buffers/cache:      27900      37804
Swap:       131064        864     130200
Note that it’s an idle box. The swap is slightly used and at 37MB free it is actually not too bad. Let’s try to squeeze a little bit more memory out from the factory setup.
MySQL is by far the biggest offender, and I have talked about how to reduce MySQL memory usage here. If you are just running simple CMS, InnoDB is probably not required — it uses more memory and a lot heavier on IO as well. We can simply use the LxAdmin’s mysql.cnf which I linked on the other blog post to get the bare-minimum MySQL running.

endor:~# cat > /etc/mysql/conf.d/lowendbox.cnf
key_buffer = 16K
max_allowed_packet = 1M
table_cache = 4
sort_buffer_size = 64K
read_buffer_size = 256K
read_rnd_buffer_size = 256K
net_buffer_length = 2K
thread_stack = 64K
As mysqld_safe script uses /bin/sh for scripting, it’s also a good idea to check whether dash is used instead of bash.

endor:~# apt-get install dash
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
Unpacking dash (from .../dash_0.5.4-12_i386.deb) ...
Processing triggers for man-db ...
Setting up dash (0.5.4-12) ...
endor:~# dpkg-reconfigure dash
Adding `diversion of /bin/sh to /bin/sh.distrib by dash'
Adding `diversion of /usr/share/man/man1/sh.1.gz to /usr/share/man/man1/sh.distrib.1.gz by dash'
endor:~# /etc/init.d/mysql restart
Stopping MySQL database server: mysqld.
Starting MySQL database server: mysqld.
Checking for corrupt, not cleanly closed and upgrade needing tables..
One thing I don’t like about Debian 5 is its default inclusion of rsyslog.  Well — it’s feature rich, but I don’t need MySQL and TCP syslog  support. Weight at 1.2MB RSS is just a bit too fat I reckon. I am not  game enough to go without a syslog daemon, so I just go for syslog-ng. Probably not the most light weight, but it’s just something I have been using for the last couple of years.

endor:~# ps -C rsyslogd v
 6441 ?        Sl     0:00      0   207 32936  1260  1.9 /usr/sbin/rsyslogd -c3
endor:~# apt-get install syslog-ng && dpkg --purge rsyslog
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
Setting up syslog-ng (2.0.9-4.1) ...
Starting system logging: syslog-ng.
(Reading database ... 16517 files and directories currently installed.)
Removing rsyslog ...
Purging configuration files for rsyslog ...
endor:~# ps -C syslog-ng v
 8215 ?        Ss     0:00      0   105  2754   708  1.0 /usr/sbin/syslog-ng -p
Shedding 500kb RSS — not too bad I guess :)
Next — Portmap and FAM got installed when Lighttpd was first installed. Lighttpd does not really need FAM. It’s used for stat cache to reduce seeks, but can live without. Not that I have noticed any performance difference anyway for small traffic anyway. Having both of them removed from the process list would give us extra 750KB.

endor:~# apt-get remove --purge portmap
eading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
OpenSSH can be replaced by dropbear to save memory.

endor:~# touch /etc/ssh/sshd_not_to_be_run
endor:~# apt-get install dropbear
endor:~# vi /etc/default/dropbear
endor:~# /etc/init.d/dropbear start
Starting Dropbear SSH server: dropbear.
Just remember to set NO_START=0 in /etc/default/dropbear so dropbear can run as a daemon. Dropbear daemon is using around 500KB less than OpenSSH daemon during idle, and for each connection it uses 1.5MB less on this Debian 5 box — that’s quite a saving!
That’s probably it! Vixie cron can be replaced by a light weight DCRON but I can’t seem to be able find it in Debian’s repository. Exim4 is probably one of the most light weight mail daemon you can have, but then again you might want to question — “do I need a mail daemon running”? You can probably bring it down, and just run /usr/sbin/runq once every 2 hours to process the queue, in case the previous delivery failed. That would probably give you another 1MB to play with.
You can also use PDKSH to replace BASH as interactive shell to loose some weight.

endor:~# ps -C bash v
 8409 pts/1    Ss     0:00      2   663  2140  1568  2.3 -bash
endor:~# apt-get install pdksh
endor:~# chsh -s /bin/pdksh

# ps -C pdksh v
 8550 pts/0    Rs     0:00      0   174  1633   588  0.8 -pdksh
That’s 1 full megabyte off the scale! Also note that VPSLink’s /etc/inittab  automatically spawn a BASH process on the console — just in case you  got locked out from firewall. For me it’s the last line of inittab file. Change it to /bin/sh or /bin/pdksh, run init q to reload init(1), and then kill that bash process.
Here’s the end result:

# ps aux
root       325  0.0  0.4   2032   292 ?        S<s  Mar17   0:00 udevd --daem
root      1216  0.0  0.0      0     0 ?        S    Mar17   0:00 [pdflush]
root      6470  0.0  1.3   2048   896 ?        Ss   Mar17   0:00 /usr/sbin/cron
103       6973  0.0  1.3   6112   912 ?        Ss   Mar17   0:00 /usr/sbin/exim4
root      7953  0.0  0.7   1716   524 ?        S    00:23   0:00 /bin/sh /usr/bi
mysql     7992  0.0  8.2  37904  5404 ?        Sl   00:23   0:00 /usr/sbin/mysql
root      7994  0.0  0.8   1636   536 ?        S    00:23   0:00 logger -p daemo
root      8215  0.0  1.1   2860   776 ?        Ss   00:31   0:00 /usr/sbin/syslo
www-data  8313  0.0  2.4   5712  1640 ?        S    00:37   0:00 /usr/sbin/light
root      8418  0.0  0.7   2052   468 ?        Ss   00:51   0:00 /usr/sbin/dropb
root      8527  0.0  0.7   1712   468 ?        Ss   01:19   0:00 /bin/sh --
root      8549  0.0  1.9   2712  1300 ?        Ss   01:21   0:00 /usr/sbin/dropb
root      8550  0.0  0.9   1808   600 pts/0    Rs   01:21   0:00 -pdksh
root      8562  0.0  1.3   2308   908 pts/0    R+   01:26   0:00 ps aux
# free
             total       used       free     shared    buffers     cached
Mem:         65704      58852       6852          0       2180      40344
-/+ buffers/cache:      16328      49376
Swap:       131064        380     130684
That’s 12MB trimmed, which can be used in disk cache to improve static file serving.


So how do we conclude? 64MB is more than enough to serve a few low traffic static websites. You can actually run a few WordPress sites with a few hundred visitors a day — at the price equivalent to many heavily oversold shared hosting and you get root access!
One thing about root access though — in all my examples above I used root account and never bothered to use a normal user account. It is bad from security aspect so don’t do it. Or at least don’t tell anyone that you use nothing but root :)