Guide to configure an XEN Dom0 server based on Alpine 3.15¶
I have the following configuration for my file server.
- Server HP ProLiant MicroServer Gen8 G1610T
- Memory Kingston KCP316ED8/8 8GB DDR3 1600MHz ECC Module, 16GB total.
- USB Boot USB 16GB Stick Cruzer
During the installation we also need an USB stick (1G+), as well as a workstation/laptop.
- Temporary USB Stick, to be used as installation media.
Here are some various references I have been looking at
This chapter should be executed on your normal desktop/laptop.
Insert the installation Media USB stick into your desktop/laptop, and check which device it is located at
# dmesg | tail ... [162562.819054] sd 5:0:0:0: [sdb] 2046240 512-byte logical blocks: (1.04 GB/999 MiB) [162562.823977] sdb: sdb1
You can also check dmesg for removable disks, as per this example.
# dmesg | grep -i removable [14607526.725995] sd 16:0:0:0: [sdb] Attached SCSI removable disk [15742096.383712] sd 17:0:0:0: [sdb] Attached SCSI removable disk #
On my system, we can see that the Installation Media USB stick has been attached as
/dev/sdb, but please note that this varies from system to system.
Please, ensure that the Installation Media USB stick has not been automatically mounted.
$ mount | grep <Removable Installation Media USB Stick> $ df | grep <Removable Installation Media USB Stick>
If it had been auto-mounted, please unmount it. Since the
dd command expects it to be unmounted.
# umount <Removable Installation Media USB Stick>
Time to move the just downloaded Alpine Xen 3.15 Image to the installation Media USB stick using
The command we should use is
dd, and syntax
# dd if=/tmp/alpine-xen-3.15.0-x86_64.iso of=<Removable Installation USB Stick>
In my case the command would be
$ sudo -i # dd if=/tmp/alpine-xen-3.15.0-x86_64.iso of=/dev/sdb 430080+0 records in 430080+0 records out 220200960 bytes (220 MB, 210 MiB) copied, 76.6593 s, 2.9 MB/s # sync
At this time, we are done working on our workstation, time to move over to the new server.
Prepare Boot USB Stick¶
Time to start the installation on the new Alpine Server. Insert the newly created Installation Media USB Stick, and reboot the server. Ensure that you boot from the installation media.
When server has booted, insert the Boot USB Stick (16GB USB Cruzer in my case)
At login prompt, login as
root, no password at this stage.
Time to figure out which device the Boot USB Stick device has.
Look in the
dmesg output and search for your USB stick. In my case Cruzer.
It should be located towards the end of the
# dmesg [ 12.296717] usb 1-1.1: new high-speed USB device number 3 using ehci-pci [ 12.313381] usb 2-1.3: new high-speed USB device number 3 using ehci-pci [ 12.398454] usb 1-1.1: New USB device found, idVendor=0781, idProduct=5571 [ 12.398457] usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 12.398459] usb 1-1.1: Product: Cruzer Fit [ 12.398461] usb 1-1.1: Manufacturer: SanDisk [ 12.398463] usb 1-1.1: SerialNumber: 4C530001231102113292 [ 12.398886] usb-storage 1-1.1:1.0: USB Mass Storage device detected [ 12.399172] scsi host8: usb-storage 1-1.1:1.0 [ 13.410934] scsi 8:0:0:0: Direct-Access SanDisk Cruzer Fit 1.00 PQ: 0 ANSI: 6 [ 13.412237] sd 8:0:0:0: [sde] 30595072 512-byte logical blocks: (15.7 GB/14.6 GiB) [ 13.414122] sd 8:0:0:0: [sde] Write Protect is off [ 13.414126] sd 8:0:0:0: [sde] Mode Sense: 43 00 00 00 [ 13.415246] sd 8:0:0:0: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 13.420719] sde: sde1 sde2 [ 13.423963] sd 8:0:0:0: [sde] Attached SCSI removable disk
In my case the device we will be using is
We need to format the Boot USB stick with two partitions
- We need a small bootable boot partition for dom0, about 1GB is enough.
- The rest will be used as a LVM partition, for holding supporting domU's
We will format using
fdisk. Make sure you target the Boot USB stick.
# fdisk /dev/sde
End result should look like this
# fdisk -l /dev/sde Disk /dev/sde: 15 GB, 15664676864 bytes, 30595072 sectors 1904 cylinders, 255 heads, 63 sectors/track Units: cylinders of 16065 * 512 = 8225280 bytes Device Boot StartCHS EndCHS StartLBA EndLBA Sectors Size Id Type /dev/sde1 * 0,1,1 122,254,63 63 1975994 1975932 964M c Win95 FAT32 (LBA) /dev/sde2 123,0,1 1023,254,63 1975995 30587759 28611765 13.6G 8e Linux LVM
NOTE observe that
sde1 is a bootable partition.
NOTE you might need to unplug/replug the stick after partitioning it.
# partprobe; sleep 2; mdev -s
Time to add the
# apk add syslinux
Load the VFAT kernel module
# modprobe vfat
Create bootable file system
# mkfs.vfat /dev/sde1
Check the UUID of the newly created filesystem, we need the UUID later so make a note of it
# blkid | fgrep sde1 /dev/sde1: UUID="61DF-01FB" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="2a6b8479-01"
We need to know which device is the Installation Media, to do this do a
df, and in my case the Installation media was mounted at
/media/sdc (it might be mounted on different place depending on your situation)
# df | grep '/media/' /dev/sdc 986036 585164 400872 59% /media/sdc
setup-bootable script to add Alpine Linux to the Boot USB stick and make it bootable (replacing
sde with your Boot USB stick name):
# setup-bootable <Installation media> <Boot USB Stick> # setup-bootable /dev/sdc /dev/sde1
This steps might take a few minutes.
setup-bootable script is finished, the installation is done, and we can remove the Installation media, and reboot.
<remove installation media> # reboot
Basic alpine host setup¶
After reboot, login on console again, (still no password for root), and it is time to configure this alpine installation.
eth0, and add
br0, end result should be like below.
- Confirm that the IP, DNS, GW are correct and valid
- Based on some script dependencies, it is best to name the bridge
- For DNS IP, please use your own, or if unsure, use Google
below is printout of my interfaces file after the
setup-alpine script was run.
auto lo iface lo inet loopback auto br0 iface br0 inet static bridge-ports eth0 address 192.168.1.5 netmask 255.255.255.0 gateway 192.168.1.1
- Which disk(s) would you like to use:
- Enter where to store configs:
- Enter apk cache directory:
After setup has finish, you need to commit the changes to the Boot USB stick.
# lbu commit
Switch to mount with UUID¶
After the installation a line with
/dev/sdb1 mounted as
/media/sdb1 was added in
/etc/fstab, the problem is that
sdb is a dynamic name so
we should switch to mounting via UUID.
/dev/sdb1 /media/sdb1 vfat ro,relatime,fmask=0022,dmask=0022,errors=remount-ro 0 0
UUID=61DF-01FB /media/flash vfat ro,relatime,fmask=0022,dmask=0022,errors=remount-ro 0 0
# mkdir /media/flash; mount /media/flash
lbu.conf and the apk
# sed -i's/sdb1/flash/' /etc/lbu/lbu.conf /etc/apk/repositories
Fix apk cache symlink
# rm /etc/apk/cache; ln -sf /media/flash/cache /etc/apk/cache
# lbu commit
Add main user¶
I took a small short cut here, and added a normal user.
This so I could SSH to the new Alpine server, and copy paste commands from this document.
Note Skip the use of
etckeeper as its not installed if taking a short-cut (use
At login prompt, login as
root, use the password selected in
... or login via ssh as the "main user" optionally created above and use
su - to switch to root
Etckeeper will store whole /etc in a git repository, and keep track of every change
that is made there.
Just remember to do a etckeeper commit "Describe the change"
# apk add etckeeper # cat << EOF > /etc/.git/hooks/post-commit #!/bin/sh set -e lbu commit EOF # chmod 755 /etc/.git/hooks/post-commit # lbu commit
First, we need to confirm what name the Boot USB stick has (in my case
Time to add and configure LVM.
# apk add lvm2 # pvcreate /dev/sde2 Physical volume "/dev/sde2" successfully created. # vgcreate vg_domU /dev/sde2 Volume group "vg_domU" successfully created # lvcreate -n lv_domU_installer -L 1G vg_domU Logical volume "lv_domU_installer" created. # rc-update add lvm * service lvm added to runlevel default # etckeeper commit "Added LVM"
And setup the filesystem.
# apk add e2fsprogs # mkfs.ext4 /dev/vg_domU/lv_domU_installer
And prepare for domU installation medias.
# mkdir /domU_installer # echo "/dev/vg_domU/lv_domU_installer /domU_installer ext4 noauto,noatime 0 0" >> /etc/fstab # mount /domU_installer # etckeeper commit "Added /domU_installer to fstab"
NOTE That the
/domU_installer is not mounted automatically on boot, but has to be specifically mounted by user when needed, as well as unmounted when not needed anymore.
When you try to create a domU, and
xl complains about a missing media is usually a sure sign that you forgot to
We need to add the
xen-hypervisor manually, since we run diskless.
# apk add xen-hypervisor
Need to ensure xendomains starts at reboot
# rc-update add xendomains # etckeeper commit "Start xendomains at reboot"
We will use
tmux to capture the various domU's console.
# apk add tmux
Then we need to uncomment a line in the
/etc/conf.d/xendomains configuration file
# etckeeper commit "Configure xendomains to use tmux"
After you have created some domU's the output might look like below, and based on the below output the
tmux session is called
# tmux ls 0: 1 windows (created Wed Sep 5 05:39:38 2018) [127x53] 4: 1 windows (created Sun Sep 9 01:24:43 2018) [127x53] xen: 4 windows (created Mon Sep 10 14:44:51 2018) [127x53]
And you would then connect to the
tmux session with
tmux attach-session using the following command:
# tmux attach-session -t xen
For further information on how to navigate a
tmux session, please check the following excellent Tmux Cheat Sheet
Prepare for domU¶
Prepare domU's installer image (needed to create you first domU later) Download Alpine Extended from link to Alpine 3.15.0 Extended 64 bits, or if You need an other version go the Alpine Download Page and download Your preferred "Extended" version.
The reason we use the
-extended flavor, is so when we create our domU;s we do not have to have network configured and protected when we do the installations.
Remember to download the checksum file as well.
# mount /domU_installer # cd /domU_installer # wget https://dl-cdn.alpinelinux.org/alpine/v3.15/releases/x86_64/alpine-extended-3.15.0-x86_64.iso.sha256 # wget https://dl-cdn.alpinelinux.org/alpine/v3.15/releases/x86_64/alpine-extended-3.15.0-x86_64.iso
After you have downloaded both ISO file as well as checksum file, you need to verify that the ISO file has the correct checksum. If not, please re-download both and try again.
# ls -lh total 598M -rw-r--r-- 1 root root 598.0M Oct 10 23:42 alpine-extended-3.15.0-x86_64.iso -rw-r--r-- 1 root root 100 Oct 10 23:39 alpine-extended-3.15.0-x86_64.iso.sha256
And now we verify that the checksum is OK.
# sha256sum -c alpine-extended-3.15.0-x86_64.iso.sha256 alpine-extended-3.15.0-x86_64.iso: OK
After we have verified that the iso image has the correct checksum, place it in the
/domU_installer directory (if not already there).
# mv <Downloaded Alpine Extended 64 ISO> /domU_installer
We prepare for the domU's installation by fetching the domU boot loader here.
# apk add grub-xenhost
The installation kernel and ramdisk we fetch from the ISO image.
# mount -t iso9660 -o ro alpine-extended-3.15.0-x86_64.iso /mnt # cp /mnt/boot/vmlinuz-lts /domU_installer # cp /mnt/boot/initramfs-lts /domU_installer # umount /mnt
Add dom0 Memory¶
Limit the memory used by the dom0 so domUs does not have to "steal" available memory from dom0, when starting.
Since dom0 is booting from the Boot USB stick, we need to update syslinux at
Previous version it was enough with 1GB ram, but I found that you could not do a system update/upgrade after fresh install with only 1G. Apparently XEN requires a bit more during upgrades, so I'm using 2G here.
# mount /media/flash -o remount,rw # vi /media/flash/boot/syslinux/syslinux.cfg
in particular, we need to add a parameter (
dom0_mem=2048M) to the row starting with
APPEND. (A lower value will not work as the dom0 is running on a RAM disk.)
APPEND /boot/xen.gz dom0_mem=2048M --- /boot/vmlinuz-lts modules=loop,squashfs,sd-mod,usb-storage quiet nomodeset --- /boot/initr
And remount Boot USB stick read only again.
# mount /media/flash -o remount,ro
And lastly we reboot and confirm the newly created Alpine Dom0 server is booting up.
# sync # reboot
When system comes up it is time to do the final touches.
- Add normal user
- Check network etc
# ping www.google.com
Lets update the system¶
# apk update # apk upgrade
For security reasons, and good practice, lets install sudo
# apk add sudo # sed -e 's/# %wheel ALL=(ALL:ALL) ALL/%wheel ALL=(ALL:ALL) ALL/g' -i /etc/sudoers # etckeeper commit "Configured sudoers"
Add a user¶
# adduser <normal-user-ID> # adduser <normal-user-ID> wheel # lbu include /home # lbu exclude /home/*/.ash_history # etckeeper commit "Added a user"
After this step, comes perhaps, and it can be argued if the ZFS host should come before the Network Domain or not...
- Installing a Basic Server as a domU.
- Installing a ZFS Storage Domain as a domU.
- Installing a Network Domain (firewall) as a domU.
- Installing a DNS Caching server as a domU.
These HowTo's are being prepared and will be linked in here later. In the mean time, have a look at the old link below.
When the package
xen-hypervisor package has been updated, the following needs to be done on the dom0 server.
- The XEN related files on the boot device (
/media/flash/boot/*xen*) needs to be updated.
After the upgrade the XEN boot related files are located in the
ramfs, so we need to copy those files to the restart safe place on the boot device
First, remount the Boot USB stick writable.
# mount /media/flash -o remount,rw
Second, copy the files needed.
# cp /boot/*xen* /media/flash/boot
Third, remount Boot USB stick read only again.
# mount /media/flash -o remount,ro
Fourth, time to verify; we need to confirm that the XEN reference in
syslinux.cfg points to the correct XEN version.
# grep xen /media/flash/boot/syslinux/syslinux.cfg APPEND /boot/xen.gz dom0_mem=2048M --- /boot/vmlinuz-lts modules=loop,squashfs,sd-mod,usb-storage quiet nomodeset --- /boot/initr
From above we see that we are using
Lets confirm that
xen.gz is the one we want. Confirming that checksum is same for
# cksum /media/flash/boot/xen*.gz 2396837473 1175423 /media/flash/boot/xen-4.15.1.gz 2396837473 1175423 /media/flash/boot/xen-4.15.gz 2396837473 1175423 /media/flash/boot/xen-4.gz 2396837473 1175423 /media/flash/boot/xen.gz
And last, confirm the above
xen.gz checksum are the same as under
# cksum /boot/xen*.gz 2396837473 1175423 /boot/xen-4.15.1.gz 2396837473 1175423 /boot/xen-4.15.gz 2396837473 1175423 /boot/xen-4.gz 2396837473 1175423 /boot/xen.gz
dom0 Kernel Upgrade¶
You can update the kernel by putting
<hostname>.apkovl.tar.gz on a freshly installed USB stick or with the
update-kernel requires more
update-kernel uses both RAM and
/tmp), hence we are temporarily
using a 4GB dedicated LVM disk for
/tmp during the kernel upgrade to be able to complete the update without running out of memory.
tempfs volume is to small for the kernel upgrade.
# df -h /tmp Filesystem Size Used Available Use% Mounted on tmpfs 155.7M 137.2M 18.5M 88% /
We add a kernel upgrade volume.
# lvcreate -n lv_kernel_upgrade -L 4G vg_domU Logical volume "lv_kernel_upgrade" created. # mkfs.ext4 /dev/vg_domU/lv_kernel_upgrade
And prepare for domU installation medias.
# mount -t /dev/vg_domU/lv_kernel_upgrade /tmp # df -h /tmp Filesystem Size Used Available Use% Mounted on /dev/vg_domU/lv_kernel_upgrade 3.9G 16.0M 3.6G 0% /tmp
Check free RAM.
# free -m total used free shared buffers cached Mem: 311 270 40 137 2 203 -/+ buffers/cache: 64 247 Swap: 0 0 0
update-kernel can be used.
# mount -o remount,rw /media/flash # update-kernel # mount -o remount,ro /media/flash
If there is a complaint on missing
mksquash-fs command, just install it, and re-run the
# apk add squashfs-tools
And we need to umount the temporary
/tmp LVM disk
# umount /tmp
Now you can reboot the dom0 to the updated kernel.
If You have troubles with nationalized keyboard (it is a bug in
setup-alpine?), then do the following work-a-round:
Into the file
/etc/profile.d/loadkeymap.sh enter the following content:
#!/bin/ash if [ "/dev/tty1" = "`tty`" ]; then echo "Loading selected keymap." /etc/init.d/loadkmap start fi
Then commit the changes.
# etckeeper commit "Added loadkeymap"
domU installation config¶
To use this installation image, you could use the below template as a base. You have to make sure that your MAC address is UNIQUE. A tool that can help you with this is, for example, random_mac.py.
If you do it manually, please start with
00:16:3E followed by a unique combination for you.
##### ##### <Hostname> domU ##### vcpus = '1' memory = '256' maxmem = '256' kernel = "/domU_installer/vmlinuz-lts" ramdisk = "/domU_installer/initramfs-lts" extra = "alpine_dev=hdc:iso9660 modules=loop,squashfs,sd-mod,usb-storage console=hvc0" disk = [ 'file://domU_installer/alpine-extended-3.15.0-x86_64.iso,hdc:cdrom,r', 'phy:<Physical Path to your disk>,xvda1,w', ] name = '<Hostname>' ## ENSURE THAT THE MAC ADDRESS IS UNIQ!!! vif = [ 'mac=<Unique MAC Address>,bridge=br0' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart'
domU running config¶
And slight modification is needed with using the proper kernel, and removing the installation image.
##### ##### <Hostname> domU ##### vcpus = '1' memory = '256' maxmem = '256' kernel = "/usr/lib/grub-xen/grub-x86_64-xen.bin" disk = [ 'phy:<Physical Path to your disk>,xvda1,w', ] name = '<Hostname>' ## ENSURE MAC ADDRESS IS UNIQ!!! vif = [ 'mac=<Unique MAC Address>,bridge=br0' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart'