VMware ESXi 3.5 /4.0 on Xen (How-to)

This may seem a truly useless idea to a lot of people, however I’ve always found having a ‘lab’ at home capable of building pretty much every system scenario very useful. Dealing daily with VMware ESX servers and VMs in a production environment means that I can never “fiddle” around and get to grips with whats under the hood or deal with the unsupported or hidden functionality. My Xen server has allowed me to create pretty much every scenario I may need Oracle RAC clusters, interoperability¬†between various operating systems and various development environments. When I first received the server that I use for my environments my first choice of setup was going to be a VMware ESX setup, however the hardware requirements restrict most installations to a subset of hardware configurations meaning I couldn’t install it. Originally it would have been impossible to install it under a xen HVM on the basis that the virtualised network adapters are unsupported by ESX, however luckily from 3.4.0 onwards the xen-tools have been updated and allow the use of the e1000/e100 network.

Step 1.

Create a basic hvm scenario:

1.5Gb LVM or disk image for the ESX install, other disks can be added for use as a datastore.

You’ll need an esx cd image, which can be downloaded along with a licence from vmware.com

I allocated about 80% of the memory of my server, along with 3 of the 4 cpus.

Set it up to output to a VNC session.

Add a nic and ensure it has the following set:


Step 2.

Start the domU up and ensure it’s booting from CD, connect to the VNC session and watch it load up the VMkernel and e1000 module etc.. Eventually you’ll get to the install steps which you can quickly go through until it errors with the issue of no devices to deploy the install to. Once at this point it’s a simple alt-F1 to the console prompt to access the ‘support’ features. At the console log in with the username root and a <blank> password, check your IDE disk is shown using fdisk -l and then cd to /usr/lib/vmware/installer/Core/ and edit with vi the file TargetFilter.py. This file is used to determine whether there is a device that can be used to deploy the ESX system to, and contains the line that we can change to allow our IDE device to be used.


return interface.GetInterfaceType() == ScsiInterface.SCSI_IFACE_TYPE_IDE


return interface.GetInterfaceType() == ScsiInterface.SCSI_IFACE_TYPE_ISCSI

Once this change has been made, the installer will be given an iSCSI device in place of in IDE device and the install will proceed to use it, this should then be kicked off by starting install from the command line. You’ll need to change back to the correct console prompt as the installer will move the terminal back, so pressing alt-F1 will return you to the correct terminal. The installer will not list your disks and the installation can begin, once completed the system will want to reboot and will come up with a DHCP address for it’s first e1000 adapter.

Regular ESX service can continue from here (i.e. get the Infrastructure Client / set [credentials,network address]  / build some VMs).

4 thoughts on “VMware ESXi 3.5 /4.0 on Xen (How-to)”

  1. I’m stuck getting no network card. Any idea from my cfg file.

    kernel = ‘/usr/lib/xen/boot/hvmloader’
    builder = ‘hvm’
    memory = ‘2048’
    shadow_memory = ‘8’
    disk = [ ‘file:/media/storage/VMdisks/ESXi/esxi-hypv1.img,hda,w’,
    name = ‘ESXi’
    vif = [‘model=ioemu, bridge=br0, mac=be:ef:de:fe:ef, model=e1000′]
    usbdevice=’tablet’ # helpful with VNC console
    on_poweroff = ‘destroy’
    on_reboot = ‘restart’
    on_crash = ‘restart’

  2. oops i pasted a typo config.

    vif = [‘type=ioemu, bridge=br0, mac=be:ef:de:fe:ef, model=e1000’]

    is the correct line. Still same problem

  3. I finally installed esxi on top of xen but i cant get any VM started I get a generic message that the vm cannt be powered on

  4. Hi Rayvyn,
    How have you installed esxi on top of xen. Can u please help me in it? I am trying to install but i get the message that no driver for interface was found. you need to install 3rd party driver. How could i do this.? Thanks in advance.
    Please help.

    >September 13, 2014

    >I finally installed esxi on top of xen but i cant get any VM started I get a generic message that the vm cannt >be powered on

Leave a Reply

Your email address will not be published. Required fields are marked *