Wednesday, February 10, 2010

Xen, Virtuozzo, Vserver, and OpenVZ

SkyHi @ Wednesday, February 10, 2010

Since 2bits moved into using VPS (Virtual Private Servers) when Xen became available we did not have to directly use other technologies, such as Virtuozzo, OpenVZ (free version of Virtuozzo), or the older User Mode Linux.

However, two recent cases proved what we have suspected for a long time: that indeed Xen is superior to competing technologies when it comes to performance.

In one case, a client contacted us for tuning a slow system. The system gets some 75,000 page views per day on weekdays, and page generation times were 2 seconds or more, and was running on a Virtuozzo host.

2bits cloned the site on a test machine ran timing benchmarks, and found that it peformed faster on the test machine. Then we cut off the external traffic from the live machine using Apache LIMIT directives, and found that page generation times have not improved. Our recommendation to the client was to move to another host. They chose to go for a dedicated server, and that solved their problems.

In a second case, someone posted a request in the forums, where he was getting page generation times of 2+ seconds, while the query components were only 700-800 ms. Upon checking the output of phpinfo, it was apparent that they used OpenVZ. The user was trusting enough to send over a copy of the site's database and files, and we installed it on a Xen VPS.

The results were page execution time of 300 to 600 ms on Xen, as opposed to 2000+ on OpenVZ.

So, the advice is: for Drupal hosting best performance: use Xen if you want a VPS. Avoid Virtuozzo and OpenVZ.

We vote for Xen we are

We vote for Xen we are running small VPS hosting company and I can say xen is the best virtuazliation ever we added http://hostbillapp.com to vps provisioning and we are reall happy for that.

This whole thing is rubbish -

This whole thing is rubbish - show benchmarks on the same hardware with the same loads.

So far the only empirical evidence presented is the HP study which says OpenVZ is faster.

I am not pro MS nor pro VZ, but in this case you have not presented anything other than circumstantial claims that Xen is better. Justify it.

Uhm you write about performance and don't mention hardware?

I've enjoyed reading other parts of this sites, but this particular article is utterly pointless because you gave zero indication what kind of hardware these servers were on.
If you compare an OpenVZ machine that shares a slow single core CPU between 10 VMs to a dedicated server with two quad cores of course the VZ is gonna loose.
Since you didn't give ANY information about hardware we have to assume that the differences you noted are entirely down to having stronger hardware in the Xen and the dedicated servers :(

Use the gearbox

Xen just can not be faster than OpenVZ/Virtuozzo, that's nonsense.

Apparently in those cases you haven't configured OpenVZ/Virtuozzo resource management properly, thus applications experienced resource shortages which is the reason of the slowdown.

Let me give you a clear example: it's like driving a sport car with manual gearbox, using only the low gear. The problem of slow car is you have to upper the gear while you increase the speed. Instead, you use a truck, and now claim that the truck is faster.

This article should help: wiki.openvz.org/Resource shortage

Your comment based on zero

Your comment based on zero personal experience with Xen and based totally on your biased assumption is the nonsense here, not that Xen is superior which it is. Nothing is superior no matter what hardware you are running than running Linux on Xen. Hardware issue is only brought up by Microsoft and VZ lovers who still hang on to old crap without even personally testing new stuff coming out which are far superior. And you point people to a crap written by VZ.org? That's absolutely laughable. Read again, Linux and Xen far superior to Microsoft and VZ.

Well, at the risk of

Well, at the risk of flogging a dead horse, I think any discussion about virutalization comparisons should reference:

http://wiki.openvz.org/Introduction_to_virtualization

My understanding is that OpenVZ/Virtuozzo is fundamentally different from Xen in how it does 'virtualization', so really, the issue is whether either technology is somehow more appropriate to Drupal. Obviously, generic comparisons can be made in favour of either, depending on your setup.

I think it would be fair to say that most of the time, there's no significant difference. I would guess that if the server's not overloaded, chances are VZ would have a slight edge. I think the key question your tests raise is: does Xen handle overload better? I'd bet your examples don't demonstrate this at all, but it's still a fair question.

From looking at your various examples, I'm going to wager that Xen degrades better because it can guarantee resources - i.e., the server can't get clobbered as easily by another virtual server on the same physical machine.

The flip side of this is that if you're lucky enough to have a VZ server on a mostly empty machine, then you can take better advantage of it.

So, if you've got issues running Drupal on a server and it's a VZ virtualization, moving it to Xen will probably work, but only because you're changing hosts!

You can not compare XEN with

You can not compare XEN with OpenVZ without detailed comparison of resources allocated to the virtual host. OpenVZ vm's by default are installed with very little resources and sometimes it is even impossible to install larger software - like java into those.

Therefore you need to examine /proc/user_beancounters and make sure that there are not any bottlenecks for your application.

Look in OpenVZ wiki for more information:
http://wiki.openvz.org/Resource_shortage

Out of interest - I noticed

Out of interest - I noticed that you said that in the second case, you installed the problem site on a Xen VPS. However it was not clear that the Xen and OpenVZ site instances were hosted and benchmarked on the same machine. If not, then I don't think you can really compare the results, since even the best resource scheduling can't make Xen on a dedicated unloaded server compete with OpenVZ on an underspecified shared server that happens to host a porn network ;)

Not a benchmark

Those were not formal benchmarks, but rather observations. So they are anecdotal evidence if you will. Jonathan Lambert seems to have observed a similar trend (see his comments).
--
2bits -- Drupal consulting

We run some OpenVZ VPSs and

We run some OpenVZ VPSs and find them very fast (well, pretty much as fast as the raw hardware...if you get better than that something is wrong!). Most things I read seem to indicate that OpenVZ has a much lower overhead, and most better resource management when under pressure (especially because of the two-level scheduler). While your results are interesting, they do seem a bit anecdotal to me :)

For example, here is part of the conclusion from the HP Xen vs. OpenVZ study.

  • For all the configurations and workloads we have tested, Xen incurs higher virtualization overhead than OpenVZ does, resulting in larger difference in application performance when compared to the base Linux case.
  • Performance degradation in Xen increases as application workloads increase. The average response time can go up by over 600% in the single-node case, and between 115% and 600% in the two-node case depending on the number of applications.
  • For all the cases tested, the virtualization overhead observed in OpenVZ is limited, and can be neglected in many scenarios.
  • For all configurations, the Web tier CPU consumptionfor Xen is roughly twice that of the base system or OpenVZ. CPU consumption of all systems and all containers goes up linearly as the workload increases. The slope of increase in the case of Xen is higher than in OpenVZ and the base cases.
  • Those who agree and those who don't

    Workhabit mention the same observation on Xen vs. Virtuozzo, in this article.

    And here are some benchmarks of Xen vs OpenVZ.
    --
    2bits -- Drupal consulting

    linux-vserver

    I moved to linux-vserver. These servers are for one organization on dedicated servers. Not vps for rent to others. So not really like the situation in your post.

    It is a chroot environment more like bsd jails. For my usage having the same kernel is not an issue. There is good support in Ubuntu and debian too.

    http://en.wikipedia.org/wiki/Linux-VServer

    Joe

    agree with you

    agree with you. I use vservers on my productive servers for some months, they now eat less system resouce compared to xen. Additionally...
    1, as for my exp.,vserver is much easier to manage.
    2, openvz is nearly the same as vserver, there's no absolute difference.

    What is the speed?

    They all work from a functionality point of view, whether it is OpenVZ/Virtuozzo, Xen, UML, or Linux Vserver.

    My point was performance. Xen is far superior over OpenVZ. How does Linux Vserver fare in this regard in your case?
    --
    2bits -- Drupal consulting

    linux-vserver performance

    I was moving from self-hosted user-mode-linux and a couple of rented vps accounts on uml too. There was a little performance lag on them but mostly a stability problem on the ones I hosted on our dedicated servers. UML development had slowed and seemed to have been leapfrogged by these others.

    I did not benchmark agains Xen but part of the decision making process was because the Linux-Vserver architecture could have performance advantages. It runs only one kernel for all virtual servers. The files in each virtual server are not in some COW like image so there isn't a slowdown there either.

    I'm going to try Virtualbox on my development desktop to see how it works.

    good subject again Khalid, thanks.

    Joe

    Agree

    I have to back up Khalid here. We are former SW-Soft Gold partners, and were involved with OpenVZ from as far back as having made the suggestion to them to not call it OpenVirtuozzo due to brand confusion. ;-)

    We have a massive cluster (several racks full) of XEN now running Drupal-based applications, and have found that in benchmarks, XEN and even VMWare bench far more reliably than OpenVZ and Virtuozzo.

    The performance question is secondary to the fact that we can get consistent results using VMWare and XEN (we use XEN now almost exclusively, VMWare only for very large clients or those with a specific requirement).

    It's the consistency that matters, because what you want to avoid is a couple of higher performance sites on a box crushing your performance when they get busy - that's something you don't get with "soft virtualization" systems like Virtuozzo, OpenVZ, or, sadly, even LVS (though proper architecture more than makes up for this with LVS). The only real sore point with Xen is the same point with any virtualization system - i/o constraint.

    This isn't really a problem until you start running intensive database-driven applications, which includes Drupal. Then you start getting into the joy of "which system-tuned variable gets crushed first" with soft virtualization systems.

    This isn't to say it can't be done, but default settings won't do it, and it's a LOT harder than just grabbing yourself a sliced up "hard virtualization" style box (XEN, VMWare, etc).

    As usual Khalid, nice comments.

    My 2c.

    Jonathan Lambert
    Principal & CEO

    WorkHabit, Inc.
    http://www.WorkHabit.com/
    http://www.WorkHabit.org/

    One advantage of Xen

    I'm far from an expert on virtualization technologies, but one Xen advantage over Virtuozzo (and maybe OpenVZ) is that you have /sys/kernel and all of the other directories that you would expect to see on under /sys on Xen. On Virtuozzo you don't see /sys/kernel and some of the others. I this has to do with how Virtuozzo shares a kernel between multiple containers on the host machine.

    A problem that I've run into is that standard Ubuntu init scripts will fail because their tests for /sys/kernel fail. This happens with the HAL init script. You can start HAL interactively and therefore you can probably script a work around and run it on boot, but a can of worms is probably opening...


    REFERENCE

    http://2bits.com/articles/hosting-virtualization-openvz-vs-xen-which-is-best.html