Why Xen is the Worst Hypervisor Out There (for me)
TL;DR: Products based on Xen are undocumented mess.
Yesterday was my second attempt to use Xen virtualization product for something useful.
A couple of years ago, I tried to use Xen for my home needs: running half a dozen of VMs for development and whatnot, like Web server, SVN server, Windows box for rare cases when I absolutely need it etc. I didn’t work out, though I tried really hard, and I am surely not a complete newbie with virtualization. I failed to make anything stable and usable of this platform. I dropped those attempts, and settled with Virtualbox. Even that typical desktop-only application turned out to be more friendly and easy to use than Xen. I held my grudge on it.
So what happened last time? I needed Xen to test another piece of virtualization software - Wind River Simics. The case of nested virtualization is in my area of interests now.
I tried to install Xenserver 6.5 - a closed source free variant of Xen source base (the open-source counterpart is called Xensource). It was terrible experience. While I managed to install the VMM itself, my numerous approaches to install a guest Linux OS inside failed. The reasons were many.
- Xen documentation is scarce and is often outdated. One has to search through the Net and to filter irrelevant pieces. There is no central authoritative place to go.
- Xen management interfaces are designed to confuse. Textual dashboard is outright useless (being only capable to show host’s IP, load and open Bash). Command-line interface is undocumented (as an exercise, find how to attach a console to a guest; there are two commands, one undocumented and another documented incorrectly, and in the end, neither worked for me). Internet is full of advices to use either xe or xl utility, xe being the new and incompatible with xl, and you can switch between them only through host reboot. Oh, and there is another deprecated xend toolset; and libvirt API. Too much of a choice is worse than no choice. At last, Xencenter GUI is the only method that allowed me to see a guest console, but it is available only for Windows (its installed failing to run in Wine) meaning you’ll need an additional host and a spare network connection to Xenserver instance to be able to carry even simple management tasks. Xencenter is also unintuitive, error messages are cryptic, like one stating “Failed to start a VM” and not bothering to explain why. Great. And of course, log files are simultaneously obscure and extra verbose to complicate grepping.
- Xen paravirtualization approach ties guest and host into an unmanageable blob. E.g., existing templates do not provide means to install vanilla Grub into PV-guest, and PV-grub or whatever it is called (I never got a chance to see if it even attempted to start) is unable to work with ext4, only with ext3. Or there was another error, I don’t know. Because PV-guest Linux installation is not officially documented. One has to crawl through numerous blog posts of varying outdatedness to no avail. HVM guests are no better. Previously, my attempt to install WinXP shattered because of unacceptable simulation speed. This time, I failed to even start HVM-guest installation, specifically, I failed to start a guest. And remember, there was no guest console, so I couldn’t see whether something even popped up on the screen. VNC connection to a guest required port forwarding, but even after that I failed to connect through vncviewer, though telnet connect to the same forwarded port showed something reasonable, like RFB protocol prompt. A trivial task of running a guest boot CD inside a VM, which is simple and well documented (and works) in Simics, Virtualbox, Qemu, ESX, VirtualPC is a rocket science in case of Xen. Mostly because PV-guest needs to know about host right from the start, even before the first boot.
- Large-enterprise-ness of Xenserver makes it really hard to perform simple tasks for a Xen beginner, which I am now, and it looks like I will always be (because it’s impossible to learn anything valuable about Xen). For example, I wanted to import an ISO image with an installer into Xenserver. It turned out there is no way to specify ISO file directly and let the VMM stash it somewhere; one will have to either create a specialized NFS/CIFS volume somewhere, or an LVM volume, or do something even more obscure. Step-by-step documentation on how to do it is absent. Good luck searching.
- Did I say that Xen documentation is missing, and/or is a mess?
I suspect this state of matters is profitable to consultants supporting Xen — they will never be without work. Me, I will keep myself as far away from Xen as possible.
Haters gonna hate.