Matt-J.co.uk : Ramblings

Hey all,

Been back in the UK since last wed morning, and really looking forward to seeing family and drinking with mates at christmas!

Work is going well since I got back, have a whole new appriciation for the GMP building after working in kista, stockholm for a few days (Sorry Hans, but our building rocks! and kst just, well, dosn’t!)

Anyway, Been trying a couple of things on the geeky side of life since I got back. One is that I finally got myself a solaris supported 8xSATA2 PCI-X card (TrendMicro with a marvell chipset) for my NAS

Can finally think about some more disks now (although I still need a new case, as this one just sucks (never spend £16 quid or less on a case :P))

I have also got round to having a look at xVM, (solaris virtualisation) (pretty much Xen.. but well integrated) I decided a few days back I would install a couple of instances of CentOS5.1 and migrate my asterix PBX and scalix MTA/Webmail/calender systems over… Not as simple as it first sounds.

Running the same OS as the guest (domU) as the main os (dom0) is pretty simple, and it’s documented all over, solaris on solaris is a peice of cake, Centos on centos, buntu on buntu, and gentoo on gentoo are all pretty easy and the steps for each can easily be found online.

However, one linux on another linux is slightly more hard work.. and then there is my corner; linux on a solaris Dom0

The idea is pretty simple,
use a spare partition, or a loopback mounted ext2 filesystem and put the linux userspace you want to run in it (first problem… all the tools for cutting out a nice copy of a userspace for centos.. are where? you guessed it, on a centos install.. Not in solaris :P)

I got round this problem with some excellent pre-made xen-ready images of the latest flavours of linux from www.jailtime.org.

Then, you simpy write a quick config showing the xen userspace tools where the OS filesystem is, and where the xen-patched kernel and initrd for the guest os are..

Next problem: I dont have a xen-patched kernel OR initrd for the guest OS.
My first guess was that they would be in the OS image under /boot, and Xen would find them when it mounted the loopback filesystem from jailtime.org.
… Not a chance.

So then I mounted the image on a linux box myself
mount -o loop -t ext2 centos51.img /centos
and had a look in the boot folder.. nothing there.

So my next cunning plan involved the fact I had already mounted this centos environment, I chrooted into it.

mount -t proc proc /centos/proc
mount -o bind dev /centos/dev
chroot /centos /bin/bash
source /etc/profile
echo nameserver x.x.x.x > /etc/resolv.conf


And from here, I used centos’s package manager to install me the xen enabled,patched kernel binary.
yum install kernel-xen

Good news! /boot suddenly had a kernel, and an initrd in it, both with xen in the name. Things were looking up.

I then copied the kernel + initrd over to my solaris box, added their paths to the xen config… and off we go again.

xm create /xvm/centos/centos1.cfg -c

…Wrong again!
This time Xen crapped out about not getting the ELF type it was expecting. It was expecting a 64Bit kernel, and it got a 32bit one.
A bit of googling was required…

Turns out the xen core 3.0 (which is the current one, and therefore shipped in solaris_nevada 77) only supports 32bit guests on 32 bit dom0’s and 64 bit guests on 64bit dom0’s (Xen 3.1 will fix this, if your CPU supports both (amd X64, X2 | intel core, pentiumD etc, but that’s too much of a waste)

Luckily moving between 64 and 32 bit in solaris is INFINATLEY easier than in linux or windows, simply a reboot with some different grub options (remove (/$ISADIR))

And sunos 5.11 is booting, in 32 bit :).. rock on!

So I try to start the vm again..
xm create /xvm/centos/centos1.cfg -c
IT BOOTS!….
Well, for about 4 seconds, then bombs out with some errors trying to mount root devices.

More googleing,
Looks like the kernel and initrd that centos’s installer gets for you when you do a ‘yum install kernel-xen’
is the kernel needed for running centos as a Xen dom0 (master) not a domU (guest) and so lacks two xen libraries in the initrd, which would allow a guest to mount shared /dev devices and access filesystem images and such.

And so that’s where im upto now, trying to find a centos xen DomU initrd, or failing that, extracting my initrd and adding the modules myself.
Will let you know how it goes.

Btw, cheers to James Legg, who got me upto speed with what he had found out about xVM so far!

–Matt