Networking and Bridges and such

Searching the internet for solutions for the strangeness created by xen’s networking solution really comes up with snippets from email chains or highly over complicated network diagrams, why? i’m not entirely sure. The default method for networking with Xen consists of a collection of ‘pokey’ scripts that seem to get (at least on my system) 90% of the way there.  I assume this again may be a ‘Gentoo’ issue however here are the steps (From a simplistic view) that are taken to create Xen networking:

  1. Original system consisted of eth0 and lo, eth0 has an ip of etc.
  2. Once the system comes up and xen starts its scripts using brctl create a network bridge, this is then used to bridge the physical interface (currently still eth0 and virtual interfaces, called vifs)
  3. xend, the xen daemon uses brctl to create a bridge called xenbr0 then things get a bit random.
  4. eth0 is renamed peth0 (peth = phystical ethernet)
  5. The ip information is taken from peth0 and peth0 is then added to xenbr0
  6. Once the peth0 is added to xenbr0 the ip information is taken from peth0 and applied to xenbr0
  7. Any xen domU that is created afterwards creates a vif which it uses, this vif is then added to the xenbr0 allowing it to communicate on the network

This is a very sparse/dumbed down version of events, however it gives an idea of whats happening. The problem that occured with Gentoo is that step 5. never happens.

What this results in is that Gentoo comes up, brings eth0 up and we have network activity for a few seconds until xen starts to get it’s claws into the network configuration. However the most simple method for repairing this involves a small configuration change in /etc/conf.d/local.start

# 00/00/00 -- IP Allocation to Xenbr0
ifconfig xenbr0 netmask
route add default gw

This is an example taken from mine, you’ll need to alter the gateway and ip address information, but put simply this will execute after every other service has been started, resulting on your domO being visible and network aware etc…

Virtualized.. Virtualised .. ?

Recently I managed to finally build and configure the Xen Hypervisor and put together a bunch of virtual machines all together. I forked out for a new box (quad core, 4Gb Ram) two months ago and getting to point that we’re at now has been a lot of tweaking, googling, patching and the odd bit of botching.

My original plan was a straight forward Vmware esx install, however it turns out that Vmware are obviously working tightly with harware vendors so it wouldnt use any old disk controller i.e. the one i have on my motherboard.

Then after a bit of deliberation it was decided that Xen would be best method to use to host virtual machines. Gentoo was selected as i decided that the ability to hand-tailor my kernel compile etc. would allow me that extra speed advantage, a bit over the top in hindsight. The build process is all well documented on the gentoo wiki -> here

Once completed the initial Gentoo install using the emerge tool took care of most of the xen install etc.. following the HOW-TO gentoo-xen guide in the wiki pretty much covers all -> here

Once the reboot was complete the box came up and then the shortfalls in the how-to became apparent:

  • the network-bridge script never worked correctly
  • finding alternate documentation is pretty hard work
  • networking once up needs tweaking for some domU’s (TCP checksums)
  • there is barely any documentation on enabling console anywhere

More to follow …