Table of Contents

Install_and_Config_Base_OS

= Initial Setup = Reboot into Hetzner's Rescue System. I was going to use Debian's AMD64 (which is not related to AMD processors!) Anyways - 32bit Etch. Configure for raid1, /boot for ext2 and / for dom0-root. PART 1 /boot ext2 256M PART 2 lvm vg0 all LV vg0 root / ext3 30G

Note: grub is required for Xen. Lilo will not work for Xen.

= Reboot into new Install = We need to add in a swap space using a neat lv (as opposed to a physical partition via the hetzner installer)

lvcreate -n dom0-swap –size 256M vg0 mkswap /dev/vg0/dom0-swap vi /etc/fstab #Add in the following: /dev/vg0/dom0-swap none swap sw 0 0

swapon /dev/vg0/dom0-swap

= Install Xen =

Check and Update Debian Package Sources

Check apt's source.list: vi /etc/apt/sources.list deb http://ftp.de.debian.org/debian/ stable main deb-src http://ftp.de.debian.org/debian/ stable main

deb http://security.debian.org/ stable/updates main #The above is a minimal config for a server located in Germany.

Main Install of Xen

Debian Etch AMD64

Note: AMD64 has nothing to do with AMD processors!! Its the name for the 64bit OS.

apt-get install linux-image-2.6.18-6-xen-amd64 xen-hypervisor-3.0.3-1-amd64 xen-tools bridge-utils sysfsutils #note libc6-xen is not required for AMD64. Note: Etch AMD64 is not just for AMD Processors! It is the name for the 64bit OS and works for Intel as well.

Debian 32bit Etch

apt-get install linux-image-2.6-xen-686 xen-hypervisor-3.0.3-1-i386-pae xen-tools bridge-utils libc6-xen sysfsutils

Check the grub boot: vi /boot/grub/menu.lst #the xen kernel should have been inserted first.

dom0_mem Boot Setting

This step is optional. If you only want to assign dom0 only 128MB of memory as opposed to allowing it to balloon out and use all memory (until a domU is created which would take the memory from dom0), you can do the following: vi /boot/grub/menu.lst #check the xen grub entry and add in the dom0_mem on the end of the kernel line: title Xen 3.0.3-1-i386-pae / Debian GNU/Linux, kernel 2.6.18-6-xen-686 root (hd0,0) kernel /xen-3.0.3-1-i386-pae.gz dom0_mem=128000 module /vmlinuz-2.6.18-6-xen-686 root=/dev/mapper/vg0-dom0–root ro console=tty0 module /initrd.img-2.6.18-6-xen-686 savedefault

dom0_mem Automatic setting

The previous step works ok. However after an apt-get upgrade for a new kernel etc. it will disappear. To have the dom0_mem always added to the grub config: vi /boot/grub/menu.lst #look for the following line: # xenhopt= change it to: # xenhopt=dom0_mem=128000 (Leave the # there.)

update-grub #update-grub scans for installed kernels and creates a new menu.lst Thanks to: http://cmrg.fifthhorseman.net/wiki/xen

Why dom0_mem

dom0_mem=128000 is needed to dedicate memory to dom0. Otherwise the following errors may be seen in /var/log/kern.log. This is with Debian Etch. It has also been seen that this squeeze error can appear after a few weeks of dom0 uptime. I highly recommend using this kernel option.

Nov 25 17:21:39 flexo kernel: printk: 4 messages suppressed. Nov 25 17:21:39 flexo kernel: xen_net: Memory squeeze in netback driver. Nov 25 17:21:44 flexo kernel: printk: 4 messages suppressed. Nov 25 17:21:44 flexo kernel: xen_net: Memory squeeze in netback driver.

Xen Network Settings

The network setup will be outlined and described later. Until then this section may be confusing to some. vi /etc/xen/xend-config.sxp #comment out the network-dummy etc. and have the following: (network-script myscript) (vif-script vif-route)

vi /etc/xen/scripts/myscript #!/bin/sh dir=$(dirname “$0”) “$dir/network-nat2” start “$dir/network-route” start chmod 755 /etc/xen/scripts/myscript

cp /etc/xen/scripts/network-nat /etc/xen/scripts/network-nat2 vi /etc/xen/scripts/network-nat2 #op_start() { # echo 1 >/proc/sys/net/ipv4/ip_forward # iptables -t nat -A POSTROUTING -o ${netdev} -j MASQUERADE # "$dhcp" != 'no' && dhcp_start #} op_start() { 

      echo 1 >/proc/sys/net/ipv4/ip_forward
      iptables -t nat -A POSTROUTING -o ${netdev} -s 10.0.0.0/24 -j MASQUERADE
      iptables -t nat -A POSTROUTING -o ${netdev} -s 192.168.1.0/24 -j MASQUERADE
      [[|"$dhcp" != 'no' ]] && dhcp_start

} #I realise that I technically should do the same above for the op_stop function.

chmod 755 /etc/xen/scripts/network-nat2

vi /etc/network/interfaces #add the following for the public ip address range auto dummy0 

iface dummy0 inet static
  address 78.1.1.20
  netmask 255.255.255.248
  broadcast 78.1.1.27

#This gives the first routable Public IP to dom0 so it can act as a gateway for the domU's. #Note the above is the quick and simple way to get networking on Xen. There is a pointopoint method allowing the use of all the public ips.

Reboot and Test

uname -a Linux root2 2.6.18-6-xen-amd64 #1 SMP Fri Jun 6 06:38:05 UTC 2008 x86_64 GNU/Linux

xm list xm top free ifconfig iptables -t nat -L ping 78.1.1.20

= Setup and Config of Xen-Tools = vi /etc/xen-tools/xen-tools.conf #below are the lines only which are uncommented: lvm = vg0 debootstrap = 1 size = 4Gb # Disk image size. memory = 128Mb # Memory size swap = 128Mb # Swap size fs = ext3 # use the EXT3 filesystem for the disk image. dist = etch # Default distribution to install. image = sparse # Specify sparse vs. full disk images. #The network will be done manually later via xm console kernel = /boot/vmlinuz-2.6.18-6-xen-amd64 initrd = /boot/initrd.img-2.6.18-6-xen-amd64 mirror = http://ftp.de.debian.org/debian/

= Automatically create a domU using Xen-Tools = xen-create-image –hostname vm01-burkesys –ip 78.1.1.21 lvscan # to see if a disk and swap lvm were created vi /etc/xen/vm01-burkesys.cfg #Make sure all is in order xm create vm01-burkesys.cfg -c #log in as root with no password passwd #change the root password vi /etc/network/interfaces gateway 78.1.1.20 netmask 255.255.255.0 ifup eth0 ping www.debian.org apt-get update apt-get upgrade

Automatically create a domU on a Private Subnet

All that changes, is that the –ip is given a 10.0.0.0 range. The config for this is done previously in the Xen Network Settings. xen-create-image –hostname vm02-priv-burke –ip 10.0.0.1 xm create vm02-priv-burke -c #login as root with no password passwd vi /etc/network/interfaces gateway 10.0.0.1 netmask 255.255.255.0 ifup eth0 ping www.debian.org #Enjoy NAT'ed traffic out.

= Migrate a domU from one Server to another =

Migrate from LVM based to LVM based Partition

On the New Xen Server (dom0)

lvcreate -n vm03-twister-disk –size 3g vg0 lvcreate -n vm03-twister-swap –size 256m vg0 mkfs.ext3 /dev/vg0/vm03-twister-disk mkswap /dev/vg0/vm03-twister-swap nc -l -p 7000 | gzip -dfc | dd of=/dev/vg0/vm03-twister-disk #if you want it faster and dont mind about bandwidth: #nc -l -p 7000 | dd of=/dev/vg0/vm03-twister-disk

On the Old Xen Server (dom0)

dd if=/dev/main-vol2/twister | gzip -cf | nc newserver.ip.address 7000 -q 10 #if you want it faster and dont mind about bandwidth: #dd if=/dev/main-vol2/twister | nc newserver.ip.address 7000 -q 10 scp /etc/xen/twister root@newserver:/etc/xen/

Thats it. Simply do a xm create on the new Xen server for the VM. Note: if the new Xen Server has a different kernel, then there may be issues. If the kernel of the VM is different on the new Server, then the /lib/modules/2.6.kernel will have to be copied onto the VM. Note: Make sure that the twister VM xen config is correct.

Migrate from File Based to LVM based Partition

In this particular case, a 2GB File based image is been copied (using dd) to a 3GB LVM Partition on a New Server. If it was only a 2GB LVM Partition, there would be nothing extra required. If the LVM partition was too small, the “dd” command would fail once the LVM filled up.

On the New Xen Server (dom0)

lvcreate -n vm04-volcano-disk –size 3g vg0 lvcreate -n vm04-volcano-swap –size 256m vg0 mkfs.ext3 /dev/vg0/vm04-volcano-disk mkswap /dev/vg0/vm04-volcano-swap nc -l -p 7000 | gzip -dfc | dd of=/dev/vg0/vm04-volcano-disk

On the Old Xen Server (dom0)

dd if=/xens/volcano/centos.5-0.img | gzip -cf | nc newserver.ip.address 7000 -q 10 #Extra steps required because the 3GB LVM is larger than the 2GB file: e2fsck -f /dev/vg0/vm04-volcano-disk resize2fs /dev/vg0/vm04-volcano-disk

Migrate a Live LVM System to a LVM based Partition

With LVM, it is possible to take snapshots of a LV (Logical Volume) which is Live, i.e. currently in use. This is extremely useful for taking backups of a live system without having any downtime. It can also be used for a barebones recovery as we are going to copy/dd all of the LV.

On the New Xen Server (dom0)

lvcreate -n vm05-zachome-disk –size 90g vg0 lvcreate -n vm05-zachome-swap –size 1g vg0 mkfs.ext3 /dev/vg0/vm05-zachome-disk mkswap /dev/vg0/vm05-zachome-swap nc -l -p 7000 | gzip -dfc | dd of=/dev/vg0/vm05-zachome-disk

On the Old Xen Server (dom0)

lvcreate -L 1G -s -n zachomebackup /dev/main-vol2/zachome lvscan dd if=/dev/main-vol2/zachomebackup | gzip -cf | nc root2.burkesys.com 7000 -q 10

Note: I hope to write a full article on LVM, how it can be used for creating snapshots etc. etc.

= Remove VMs from dom0 =

Automatically with Xen-Tools

Typically when xen-create-image is used to create a Xen VM, it creates the LVM partitions etc. and also a key config file: /etc/xen/nameofvm.cfg Its by the file extension .cfg that it will use to produce a list when xen-list-images is called. #xen-list-images #^ by default lists vms which were created with xen-tools xen-delete-image nameofvm

#if however you created the lvm and xen config manaully, you can do the following to add a .cfg extension so xen-tools will recoginise it. mv /etc/xen/nameofvm02 /etc/xen/nameofvm02.cfg xen-list-images xen-delete-image nameofvm02.cfg So xen-list-images parses all .cfg files in /etc/xen/. Whats nice is that when xen-delete-image is called, it also parses .cfg files in /etc/xen/ picks up the name of the lvm's they are using and removes them cleanly. It just saves from having to do a manual lvmremove for vms created manually.