Tuesday, April 27, 2010

Easily develop and deploy web applications from subversion

SkyHi @ Tuesday, April 27, 2010

Updated 2008-28-10.
Proper version control is a must for everyone who programs more than a
few lines of code. Even if you develop your applications all by
yourself it is very handy to be able to branch and merge your code, be
able to roll back to previous versions or undo changes you made in the
past. It works great for regular applications, but managing web
applications or websites is a tad harder for two reason: You need a
webserver to get your application going and you usually have to manage
database revisions as well.



Keeping database revisions in sync with your code revisions is a
complex subject that I will leave until another time. In this article I
will show you how you can configure your own computer or development
server in such a way that checking out or deploying a web application
is just as easy as any other piece of code.



First I will show you how to configure Apache
on your development server so that it picks up your checked out working
copies as separate subdomains. Using this, you can simply make a
checkout of your project and it will automagically be up and running.
No need to touch the Apache configuration. After that I will show you
how to use dnsmasq
so you can achieve the same effect on your own development machine.
That way you can develop your web applications locally and you won't
need a central development server. In my examples I will be assuming
you use subversion for your version control, but it works virtually the
same with other version control packages, such as git or bazaar.



Creating subdomains for every working copy



Here is what we want to achieve. Suppose that you have a development server at http://dev.example.org
where you develop your websites. We want that every working copy is
it's own subdomain on that server. If I check out a working copy and
name it “foobar” then it should be available on http://foobar.dev.example.org. In order for this to work, you need a wildcard DNS entry so that all subdomains of dev.example.org
go to your development server. I won't cover how you do that. If you do
not run your own DNS server, ask your network administrator about it or
use dnsmasq as explained in the second part of this article.



Start by creating a directory where all your working copies will live. For this article I will assume that this is /home/checkout. Every directory under this directory will become a subdomain on your development server. Create a trunk directory under the checkouts directory (/home/checkout/trunk). This is where the main dev.example.org
domain will go to. When a subdomain does cannot resolve to a directory
under /home/checkouts we want to show an error page. I created /home/checkouts/error.html containing some useful information, but you could also redirect it to a 404 error if you want.



Then create a new virtual host configuration for Apache.



  1. <VirtualHost *:80>
  2.         DocumentRoot /home/checkout/trunk
  3.         ServerName dev.example.org
  4.         ServerAlias *.dev.example.org
  5.  
  6.         LogLevel warn
  7.         ErrorLog /var/log/apache2/error.log
  8.         CustomLog /var/log/apache2/access.log combined
  9.         ServerSignature On
  10.  
  11. </VirtualHost>


We will use mod_rewrite to tell Apache about all the subdomains. The
following mod_rewrite rules extract the subdomain from the request, see
if a directory exists in /home/checkout that matches the subdomain and then redirects the request to that directory (hat-tip to Stuart).



  1.         RewriteEngine On
  2.  
  3.         # If the request contains a subdomain and the directory exists, redirect
  4.         RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  5.         RewriteCond /home/checkout/%1 -d
  6.         RewriteRule ^(.*) /home/checkout/%1/$1 [L]


Next, we add some more rules to redirect any non-existing subdomains to the error page:



  1.         # Redirect non-existing subdomains to the error page
  2.         RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  3.         RewriteRule ^(.*) /home/checkout/error.html [L]


Update: Kevin points out in this comment that using the VirtualDocumentRoot directive is an even nicer solution than these rewrite rules.



Finally we want to hide the .svn directories that subversion creates
when you make a new checkout. If you use a different version control
system such as git, then replace .svn with .git in the configuration
below:



  1.         # Hide .svn directories in checkouts
  2.         <Directory ~ "/\.svn/">
  3.                 order allow,deny
  4.                 deny from all
  5.         </Directory>


Restart Apache and you're done. Now you can simply log into the development server, make a new checkout in the /home/checkout directrory and you can immediately browse to it. After executing the example below, you can simply browse to http://my-working-copy.dev.example.org and see your website.



  1. $ ssh dev.example.org
  2. Last login: Wed Aug  6 09:36:33 2008 from 10.0.0.99
  3.  
  4. dev$ cd /home/checkout
  5. dev$ svn checkout https://svn.example.org/project/trunk my-working-copy


The full Apache configuration file now looks like this (download):



  1. <VirtualHost *:80>
  2.         DocumentRoot /home/checkout/trunk
  3.         ServerName dev.example.org
  4.         ServerAlias *.dev.example.org
  5.  
  6.         LogLevel warn
  7.         ErrorLog /var/log/apache2/error.log
  8.         CustomLog /var/log/apache2/access.log combined
  9.         ServerSignature On
  10.  
  11.         RewriteEngine On
  12.  
  13.         # If the request contains a subdomain and the directory exists, redirect
  14.         RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  15.         RewriteCond /home/checkout/%1 -d
  16.         RewriteRule ^(.*) /home/checkout/%1/$1 [L]
  17.  
  18.         # Redirect non-existing subdomains to the error page
  19.         RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  20.         RewriteRule ^(.*) /home/checkout/error.html [L]
  21.  
  22.         # Hide .svn directories in checkouts
  23.         <Directory ~ "/\.svn/">
  24.                 order allow,deny
  25.                 deny from all
  26.         </Directory>
  27.  
  28. </VirtualHost>


Developing locally with dnsmasq



Developing your websites on a central development server works, but
most often it's easier to develop on your local machine instead. It's
usually faster and there are quite a few handy tools out there that
don't work transparently over an ssh connection. When you have a Linux
desktop or laptop it's dead easy to do. Simply install Apache, PHP,
MySQL or whatever server-side tools you need on your desktop, copy the
configuration files from the server and modify them.



The only problem with using the above Apache configuration locally
is the DNS wildcard. Unless your desktop is assigned a hostname by your
network's DNS server and you can set the wildcard there, you will have
to make do with your localhost address. You can install dnsmasq
to act as a local caching DNS server and put the wildcard on your own
machine. As a bonus, the DNS caching can also speed up your web
browsing (hat-tip to Carthik).
If you want, you can replace localhost with another name (like your
hostname) in the commands below. The commands below are for Debian
Lenny, but should be similar on other distributions. Let's start my
installing dnsmasq.



  1. # apt-get install dnsmasq


Now we edit /etc/dnsmasq.conf So that it listens on 127.0.0.1 for our DNS requests and tell it to point all subdomains of localhost to 127.0.0.1. Near line 92 you will find the listen-address configuration, which tells dnsmasq to listen for DNS requests. Add the following line:

  1. listen-address=127.0.0.1


You can force domains to resolve to a certain IP with the address
directive, which you find near line 63. Add the following line to make
localhost have all it's subdomains pointing to 127.0.0.1:



  1. address=/localhost/127.0.0.1


Most likely your machine connects to the network via DHCP and gets
it's nameservers from the DHCP server. We need to configure DHCP so
that it always asks the local dnsmasq server first before trying the
other DNS servers. Open up /etc/dhcp3/dhclient.conf and uncomment the following line:



  1. prepend domain-name-servers 127.0.0.1;


The next time you connect to the network, the DHCP client will add 127.0.0.1 as a nameserver to /etc/resolv.conf,
before the other nameservers. If you do not want to reconnect to the
network to get this change, you can add it manually for now. Open up /etc/resolv.conf and add the following line before the nameserver directives, but after the search directive:



  1. nameserver 127.0.0.1


Restart dnsmasq for the changes to take effect:



  1. # /etc/init.d/dnsmasq restart
  2. Restarting DNS forwarder and DHCP server: dnsmasq.


Now you can configure Apache according to the first part of this article, replacing dev.example.org with localhost. Do not forget to remove Apache's default virtual host, which also listens on localhost.



  1. # rm /etc/apache2/sites-enabled/001-default


Reload Apache and now you can simply make a checkout of your website
to your local machine and immediately view it in your browser by going
to my-working-copy.localhost. Happy coding!

References


  1. http://en.wikipedia.org/wiki/Wildcard_DNS_record
  2. http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html
  3. http://muffinresearch.co.uk/archives/2006/08/20/redirecting-subdomains-to-directories-in-apache/
  4. http://ubuntu.wordpress.com/2006/08/02/local-dns-cache-for-faster-browsing/
  5. http://www.jejik.com/articles/2008/08/easily_develop_and_deploy_web_applications_from_subversion/#18