Introduction

Welcome, reader, to another Mind technical article!

This article is the second in a series covering the steps involved in setting up an A/B boot and update scheme with qemu-x86_64. The first article covered the basics with Buildroot.

This alone was not sufficient, and so in this second part, we will take a look at what is missing. More specifcally, we will develop the scripts defining the required partition layout, implement the A/B selection logic in a U-Boot script, detail the configuration for RAUC and, finally, customize the Linux kernel’s build options.

Scripts for A/B Boot

Buildroot uses several stages of scripting to fine-tune the software artifacts that are flashed to an embedded device.

We use a build-time post-image script to generate the A/B partition layout and a suitable U-boot environment. A post-build script ensures this U-boot environment is accessible from the Linux OS. Another dedicated U-boot run-time script implements the boot partition selection logic.

The Post-Image Script

The post-image script created in the first blog post during Buildroot configuration was essentially empty. This script is called right after Buildroot has finished generating the file system image. In our case, it should generate the U-boot environment and set up the image partition layout defined in a genimage.cfg configuration file that we will flesh out later.

Its contents should be put in a blog-dir/board/qemu-x86_64-AB/post-image.sh file:

#!/bin/bash

BOARD_DIR="$(dirname $0)"
BOARD_NAME="$(basename ${BOARD_DIR})"


main() {
	#find uboot-tools
	pushd ${BUILD_DIR}
	U_BOOT_TOOLS=$(find -maxdepth 1 -type d -name "host-uboot-tools-*")
	if [ -z "$U_BOOT_TOOLS" ]; then
		echo "ERROR: Could not find host uboot-tools"
		exit 1
	fi
	U_BOOT_TOOLS=$(realpath $U_BOOT_TOOLS/tools)
	popd


	#Buildroot compiles the script automatically, and outputs it in host-uboot-tools
	#To refresh the script, rebuild host-uboot-tools.
	cp $U_BOOT_TOOLS/boot.scr ${BINARIES_DIR}/boot.scr

	#Build a filesystem for our compiled script
	mkdir -p ${BUILD_DIR}/tmproot
	cp ${BINARIES_DIR}/boot.scr ${BUILD_DIR}/tmproot/
	rm -f ${BINARIES_DIR}/boot.ext2
	mkfs.ext4 -d ${BUILD_DIR}/tmproot -t ext2 -r 1 -N 0 -m 5 -L "boot" -I 256 -O ^64bit,^metadata_csum ${BINARIES_DIR}/boot.ext2 "1M"
	rm -r ${BUILD_DIR}/tmproot


	#Generating the image according to the layout given in the genimage.cfg
	support/scripts/genimage.sh -c "${BOARD_DIR}/genimage.cfg"
	exit $?
}

main $@

We disable the metadata_csum option when generating boot.ext4 to ensure compatibility with U-boot’s ext file system driver. The genimage.cfg configuration file mentioned earlier is given below. Its content should be put in the blog-dir/board/qemu-x86_64-AB/genimage.cfg file.

image qemu.img {
	hdimage {
		partition-table-type = "gpt"
	}
	partition ubootdata {
		image = "boot.ext2"
		size = 1M
	}
	partition rootfs1 {
		image = "rootfs.ext4"
	}
	partition rootfs2 {
		image = "rootfs.ext4"
	}
}

The Post-Build Script

The post-build script helps us modify the target file system before it is converted to a file system image. Indeed, to access U-boot’s environment from Linux, we need to mount the U-boot data partition during Linux’s boot. This is achieved by adding a line to the /etc/fstab file of the target’s file system with the following script, this should be put in the file blog-dir/board/qemu-x86_64-AB/post-build.sh.

#!/bin/bash

BOARD_DIR="$(dirname $0)"
BOARD_NAME="$(basename ${BOARD_DIR})"


main() {
    # Append instructions to auto-mount the environment partition
    MOUNT="/dev/vda1       /mnt/uboot      ext2    rw,defaults     0       0"
    if grep -Pq "$MOUNT" "$TARGET_DIR/etc/fstab"; then
        echo "Mounting uboot env already enabled"
    else
        echo "$MOUNT" >> "$TARGET_DIR/etc/fstab"
    fi
}

main $@

The U-boot Script

In the RAUC documentation we can find an example of a U-boot boot script for an A/B boot scheme. We use this as a starting point for our own script.

Because we use a virtualised disk and a specific partitioning layout with qemu , this scripts requires changes for the definitions of the A and B partitions: From the partition layout we can deduce that the partition numbers are 2 and 3. The storage interface connecting the virtual disks to U-boot is virtio.

Changes with Reference

test -n "${BOOT_A_DEV}" || setenv BOOT_A_DEV "virtio 0:2"
test -n "${BOOT_B_DEV}" || setenv BOOT_B_DEV "virtio 0:3"

We must also specify where U-boot should search for the kernel images on each partition, but be careful with the kernel image file: its name and compression varies with the target system.

setenv load_kernel "load ${BOOT_A_DEV} ${kernel_addr_r} /boot/bzImage"
setenv bootargs "${default_bootargs} root=/dev/vda2 rauc.slot=A"

setenv load_kernel "load ${BOOT_B_DEV} ${kernel_addr_r} /boot/bzImage"
setenv bootargs "${default_bootargs} root=/dev/vda3 rauc.slot=B"

We also need to specify the default Linux kernel launch arguments:

setenv default_bootargs "rootwait console=ttyS0 debug earlyprintk=ttyS0,115200"

The last step is to configure U-boot’s boot command. Beware that it may differ based on the target.

zboot ${loadaddr_kernel}

Complete U-boot Script

The complete U-boot script is put in ../board/qemu-x86_64-AB/ab-uboot.script and has this contents:

test -n "${BOOT_ORDER}" || setenv BOOT_ORDER "A B"
test -n "${BOOT_A_LEFT}" || setenv BOOT_A_LEFT 3
test -n "${BOOT_B_LEFT}" || setenv BOOT_B_LEFT 3

test -n "${BOOT_A_DEV}" || setenv BOOT_A_DEV "virtio 0:2"
test -n "${BOOT_B_DEV}" || setenv BOOT_B_DEV "virtio 0:3"

setenv bootargs
setenv default_bootargs "rootwait console=ttyS0 debug earlyprintk=ttyS0,115200"
for BOOT_SLOT in "${BOOT_ORDER}"; do
  if test "x${bootargs}" != "x"; then
    # skip remaining slots
  elif test "x${BOOT_SLOT}" = "xA"; then
    if test 0x${BOOT_A_LEFT} -gt 0; then
      echo "Found valid slot A, ${BOOT_A_LEFT} attempts remaining"
      setexpr BOOT_A_LEFT ${BOOT_A_LEFT} - 1
      setenv load_kernel "load ${BOOT_A_DEV} ${kernel_addr_r} /boot/bzImage"
	  setenv bootargs "${default_bootargs} root=/dev/vda2 rauc.slot=A"
    fi
  elif test "x${BOOT_SLOT}" = "xB"; then
    if test 0x${BOOT_B_LEFT} -gt 0; then
      echo "Found valid slot B, ${BOOT_B_LEFT} attempts remaining"
      setexpr BOOT_B_LEFT ${BOOT_B_LEFT} - 1
      setenv load_kernel "load ${BOOT_B_DEV} ${kernel_addr_r} /boot/bzImage"
	  setenv bootargs "${default_bootargs} root=/dev/vda3 rauc.slot=B"
    fi
  fi
done

if test -n "${bootargs}"; then
  saveenv
else
  echo "No valid slot found, resetting tries to 3"
  setenv BOOT_A_LEFT 3
  setenv BOOT_B_LEFT 3
  saveenv
  reset
fi

echo "Loading kernel"
run load_kernel
echo " Starting kernel"
zboot ${loadaddr_kernel}

From now on, if we need to recompile the U-boot script after a build, we can use the following command from the br-qemu-x86_64 directory:

make host-uboot-tools-rebuild world

We have implemented most of the logic behind an A/B boot scheme, but not yet all of it: We still have to configure more features of U-boot itself.

U-boot Build Options

To access U-boot build options, we use

make uboot-menuconfig

There are a few things to configure here: The U-boot persistent environment options and the U-boot boot options.

_{^{NOTE: Like in the previous blog post, we display both the symbol of the configuration options and their corresponding path in uboot-menuconfig.}}

Environment >  Environment is in a EXT4 filesystem = y
#ENV_IS_IN_EXT4=y

#This is available when the above selection is done
Environment > Environment is not stored = n
#ENV_IS_NOWHERE=n

Environment >  Name of the block device for the environment = virtio
#ENV_EXT4_INTERFACE=virtio

Environment >  Device and partition for where to store the environemt in EXT4 = 0:1
#CONFIG_ENV_EXT4_DEVICE_AND_PART=0:1

Command line interface > Shell scripting commands > setexpr = y
#CMD_SETEXPR=y

The other environment options should be left to their default values. In the previous blog post, we instructed the post-build script to integrated the compiled U-boot script into the disk image. At this point, U-Boot does not yet automatically execute the script but this can simply be resolved through uboot-menuconfig.

Boot options > Enable a default value for bootcmd >  bootcmd value = echo 'Hello from A/B boot Qemu';load virtio 0:1 $loadaddr boot.scr; source $loadaddr;

#BOOTCOMMAND=echo 'Hello from A/B boot Qemu';load virtio 0:1 $loadaddr boot.scr; source $loadaddr;

Finish the Build

The basics are now configured. We can either wait for the build we have started at the end of the last blog post to finish or we can start a new one by using the following command from the br-qemu-x86_64 directory:

make -j

After the build has finished, we make sure that everything related to U-boot is recompiled and has the latest changes:

make host-uboot-tools-rebuild uboot-rebuild world

We run this command every time after we have changed the U-boot configuration.

Test the Device

Now that our disk image and U-boot are built, it is time to test everything! We cd to the blog-dir and create a vm-start.sh script to start qemu with the appropriate options. We save this code to this script:

#!/bin/bash
VM_NAME="qemu-x86_64-ab"

echo "Copying new image"
DISK_SRC=$(realpath br-qemu-x86_64/images/qemu.img)
DISK_CPY=$(mktemp -d --suffix=$VM_NAME)
#delete old copies of the disk
rm -rf "/tmp/*$VM_NAME"
#We create a temporary disk-image, this removes the need
#to regenerate a clean disk image with Buildroot each time
cp "$DISK_SRC" "$DISK_CPY"

DISK="$DISK_CPY/qemu.img"

#Find U-boot rom
UBOOT=$(realpath br-qemu-x86_64/images/u-boot.rom)
QEMU_ARGS=(
    -name "$VM_NAME"
    -smp sockets=1,cores=1
    -m 128
    -overcommit mem-lock=off
    -rtc base=utc,driftfix=slew
    -k fr-be #or whatever you want ;)
    -nographic
    -device virtio-rng-pci
    -object rng-random,id=objrng0,filename=/dev/urandom
    -drive id=disk0,file=$DISK,if=none,cache=directsync,format=raw,snapshot=off
    -device virtio-blk-pci,drive=disk0
    -bios $UBOOT #This is important, this allows us to use our own bootloader
    -machine pc
    -netdev user,id=veth0
	  -device driver=virtio-net,netdev=veth0
)
echo "qemu-system-x86_64 ${QEMU_ARGS[@]}"
qemu-system-x86_64 "${QEMU_ARGS[@]}"

For convenience we then make the script executable:

chmod +x vm-start.sh

We can now test the built system by running this script:

./vm-start.sh

Please note that when starting the qemu virtual machine, warnings like the following may be in the output:

Loading Environment from EXT4... ** File not found /uboot.env **

This occurs on the first boot because U-boot cannot find a saved environment since we have not saved one yet. As a consequence, U-boot loads a builtin default one, which is then used, modified and saved by our U-boot script. A next boot of the same virtual machine from the same image will show that it loads correctly.

If everything works correctly, we have the following initial output:

U-boot boot

After waiting a few more seconds, a login prompt is shown. Because users and no password were configured, typing root is sufficient to login into the virtual device.

We can now see the A/B boot status according to rauc. We use the following command:

rauc status

Which gives the following output:

U-boot boot

This error message results from the fact that we haven’t configured the A/B boot logic within our Linux system!

For this, we need to configure RAUC, and reconfigure and recompile the Linux kernel…

This will be covered in the next article of this series!

Conclusion

We covered all the scripts and configuration in detail, necessary to build and run an A/B boot system in a virtualised environment through qemu.

But this is not the end yet…

We still need to create and distribute the verification keys to authenticate the update bundles… These bundles themselves also still need to be configured and built! And because we are rigorous, dedicated engineers, we need to verify that it all works by testing…

Even given the fact that we are nearing the end, the remaining work exceeds the scope of this post, so we will continue in a third and final part.

Until then, try to survive the summer heat!

Simulating an A/B update scheme using qemu-x86: Part 2