KVM stands for Kernel-based Virtual Machine and is my preferred way of virtualizing a Linux system, in combination with libvirt. This article documents a few hints on what my current best practice setup is.
For the host and most VMs, I use Debian wheezy (which is not yet released as of writing this article). I want a recent kernel and, most importantly, systemd.
An easy setup is to just use a bridge which contains the real ethernet device (eth0). However, I have noticed a few problems with IPv6 routes on bridge devices which turned out to be kernel bugs (1, 2, possibly I forgot some). The symptom is similar to this post by Marc, essentially leading to VMs which don’t come up properly after rebooting the host or even rebooting just the VM.
Therefore, I use
tap devices instead. They are a bit complicated
to setup. See the libvirt documentation on generic ethernet connections and this libvirt wiki page, which explains how you can lower the host protection to be able to use these interfaces. Here is the XML part of the VM definition which I use:
<interface type='ethernet'> <mac address='52:54:00:fa:f1:f1'/> <script path='/etc/libvirt/qemu/networks/ifup-infra.sh'/> <target dev='tap.infra'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </interface>
#!/bin/sh # # Network configuration for kvm domain "infra" # # Generate a random (but fix) MAC address: # echo -n 'f6:'; openssl rand -hex 5 | sed 's/\(..\)/\1:/g; s/.$//' # F6 is a prefix which is considered Locally Administered because the second # lowest bit of the first byte is set to 1. set -e /sbin/ip link set $1 address f6:63:cb:bb:59:e1 /sbin/ip link set $1 up /sbin/ip -4 route add 22.214.171.124/32 dev $1 /sbin/ip -4 route add 126.96.36.199/32 via 188.8.131.52 dev $1 /sbin/ip -6 route add 2001:4d88:100e:1::/64 dev $1 # RZL VPN /sbin/ip -6 route add 2001:4d88:100e:ccc0::/64 via 2001:4d88:100e:1::2 dev $1 /sbin/ip -6 route add 2001:4d88:100e:ccc::/64 via 2001:4d88:100e:1::2 dev $1
The script sets a static, locally administered MAC address on the tap interface on the host. This is necessary so that the VM can use the same link-local address as default route.
On the host, I have a /27 IPv4 network and a /48 IPv6 network. Previously, I used an IP address configuration such as 184.108.40.206/27 inside the VM. However, that only works when you have no special routes such as an IP address you route into a VPN (or you add routes within each VM which is error-prone and tiresome).
Therefore, I configure IPs in the VM with a /32 netmask, e.g. 220.127.116.11/32. The default IPv4 route within each VM is 192.168.23.23, so that I don’t lose one of my precious public IPs for that purpose:
host # ip -4 address add 192.168.23.23/32 dev lo vm # ip -4 route add 192.168.23.23 dev eth0 vm # ip -4 route add default via 192.168.23.23
For IPv6, we use a link-local address, which depends on the MAC address we set earlier in
vm # ip -6 route add default via fe80::f463:cbff:febb:59e1 dev eth0
For the reference, here is the entire
/etc/network/interfaces of a VM:
auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet static address 18.104.22.168 netmask 255.255.255.255 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 22.214.171.124 dns-search in.zekjur.net post-up ip -4 route add 192.168.23.23 dev eth0 post-up ip -4 route add default via 192.168.23.23 post-up ip -4 address add 126.96.36.199/32 dev eth0 post-up ip -6 address add 2001:4d88:100e:1::2/64 dev eth0 post-up ip -6 address add 2001:4d88:100e:1::3/64 dev eth0 post-up ip -6 route add default via fe80::f463:cbff:febb:59e1 dev eth0 # iptables post-up iptables-restore < /etc/network/iptables post-up ip6tables-restore < /etc/network/ip6tables
It is benefical to be able to use the
virsh console so that you
don’t have to use VNC to access your virtual machine’s text consoles. For VNC,
you probably need to create an SSH-tunnel first and then you might have issues
with your keyboard layout. Also, the virsh console is much faster since it’s
text-only. Either way (virsh console or VNC) can be used to recover your VM in
situations where you e.g. messed up
the VM is not reachable over the network anymore.
Luckily, with systemd inside the VM, the only thing you have to do is modify
/etc/default/grub like this:
GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0 init=/bin/systemd"
The order is important here! See also Lennart’s blog post about serial consoles with systemd.
In case you are not yet using systemd, you have to uncomment the getty entry
for ttyS0 in
/etc/inittab to get a login prompt.
After configuring your VM accordingly, you can use
infra to access the VM called "infra" on serial console level.
Caching / Performance
Depending on your use-case, you might want to change the cache settings which can bring dramatic disk bandwidth improvements in the virtual machine at the expense of data security.
Also see the section "Performance Tuning" of Michael David’s post about his KVM setup for further tips on performance.
I previously wrote about snapshots to backup virtual machines and still use that approach. Unfortunately, there is a bug which prevents lvremove from working properly.
I updated my script to include a workaround which works in the vast majority of cases (I only once had to clean up block devices manually in a high-load scenario since deploying the workaround).
Find it at the xen-lvm-snapshot git repository.