Configuring a Debian based XEN Network Driver domain.
Note: Tested with Debian 8 Jessie as dom0 and as Network Driver Domain domU.
Setup for a driver domain (preparations in dom0)¶
Export at least one PCI device to the new network driver domain, in this example called int-fw
dom0 # lspci | fgrep Ethernet 03:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 03:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe
Add the following to the end of /etc/xen/int-fw.cfg to export the second nic to int-fw
# BCM5720 (nic 2) pci = [ '03:00.1,permissive=1' ]
Note: permissive=1 should only be used if it does not work without it, so test without first.
Hand the device over to the dom0 xen-pciback module
dom0 # echo xen-pciback >> /etc/modules dom0 # modprobe xen-pciback dom0 # xl pci-assignable-add 03:00.1
Start the domU and install xen hotplug scripts
dom0 # xl create -c /etc/xen/int-fw.cfg
Setup within the network driver domain domU¶
root@int-fw:~# apt-get install xen-utils-common vlan bridge-utils root@int-fw:~# apt-get install --no-install-recommends xen-utils-4.4 root@int-fw:~# systemctl disable xen.service root@int-fw:~# systemctl disable xendomains.service
Setup VLANed interfaces in /etc/network/interfaces, example where the pci exported nic is eth1
auto eth1.13 iface eth1.13 inet static address 192.168.3.1 netmask 255.255.255.0 network 192.168.3.0 broadcast 192.168.3.255
Setup bridges for other domUs (e.g. a nfs domU)
auto br_nfs iface br_nfs inet static bridge_ports none bridge_stp off bridge_maxwait 0 bridge_fd 0 address 192.168.12.1 netmask 255.255.255.0
Add this to /etc/rc.local (on int-fw) to catch domUs starting while int-fw is starting at the same time.
if [ ! -e "/proc/xen/capabilities" ]; then mount -t xenfs xenfs /proc/xen || exit 1 fi export SUBSYSTEM=xen-backend export DRIVER=vif export XENBUS_TYPE=vif export ACTION=online export XENBUS_BASE_PATH=backend for vif in $(ifconfig -a | awk '/^vif/ {print $1}') do x=$(echo $vif | sed 's/^vif//' | sed 's/.0$//') y=$(echo $vif | sed 's/^vif[0-9]*\.//') export DEVPATH=/devices/vif-$x-$y export XENBUS_PATH=backend/vif/$x/$y export vif=$vif /etc/xen/scripts/vif-bridge online done
Setup of other domU:s¶
To make the other domU:s use the network driver domain instead of dom0 for networking, change
the vif line in the domU.cfg, by adding ,backend=int-fw to the end. Example:
# # Networking # vif = [ 'ip=192.168.12.10, mac=00:16:3E:xx:yy:zz, bridge=br_nfs, backend=int-fw' ]
Limitations¶
There is a need to reboot other domUs after network driver domain reboot.
The xen-utils are usually installed in a dom0 and when a dom0 is rebooted so are the domUs, hence it (at least currently) does not re-hotplug network interfaces for domUs after a network driver domain has been re-booted.
Reboot of the other domUs can be done within each of them, or preferably in the dom0 in one go:
dom0 # service xendomains restart
This also means that you must make sure that the network driver domain is the first domain that is started after a dom0 reboot.
In order to prevent needrestart from restarting xendomains by default, override service default selection to a 0, by adding the following to /etc/needrestart/needrestart.conf.
$nrconf{override_rc} = { .... # xendomains q(^xendomains) => 0, ....