Raspberry pi 4, Ubuntu 22.04, LUKS with fallback

Please find below steps for RPI4 full disk encryption (FDE) under Ubuntu 22.04 LTS using LUKS with fallbacks.

Apply LUKS

Apply to your sdcard Ubuntu Server or Desktop.

Borrowing from:

  1. raspberrypi - LUKS Disk Encryption on Raspberry Pi 4 and Ubuntu Desktop 20.10 - Ask Ubuntu.
  2. LUKS on Raspberry Pi | LUKS-on-Raspberry-Pi

Steps are repeated and translated below.

Prepare your SD card but do not install into the RPI4 – keep it on your linux desktop.

You have to be booted into a linux PC or laptop for this procedure - ideally also a matching Ubuntu version. If you don’t have that, you can follow this guide using two sdcards:

  1. Image Ubuntu onto the first sdcard, boot and log into it.
  2. Use it to image your second sdcard (via USB sdcard reader), which will become your eventual encrypted disk.

Ensure the sdcard is unmounted.

ubuntu@ubuntu:~$ lsblk 
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS 
loop0         7:0    0  57.9M  1 loop /snap/core20/1590 
loop1         7:1    0  59.1M  1 loop /snap/core20/1826 
loop2         7:2    0  71.8M  1 loop /snap/lxd/22927 
loop3         7:3    0 109.6M  1 loop /snap/lxd/24326 
loop4         7:4    0  43.2M  1 loop /snap/snapd/17954 
sda           8:0    1 183.3G  0 disk 
├─sda1        8:1    1   256M  0 part 
└─sda2        8:2    1 183.1G  0 part 
mmcblk0     179:0    0  29.8G  0 disk 
├─mmcblk0p1 179:1    0   256M  0 part /boot/firmware 
└─mmcblk0p2 179:2    0  29.6G  0 part / 

Note in this case, /dev/mmcblk0 is the temporary OS we’re booted into and /dev/sda is the target image we’ll encrypt. Store the device name (e.g. sda) in a variable to avoid accidents.

read TARGET_DEVICE
export TARGET_DEVICE 

Ensure partitions are unmounted.

sudo umount /dev/${TARGET_DEVICE}1 
sudo umount /dev/${TARGET_DEVICE}2 

Check the second partition for errors first.

sudo e2fsck -f /dev/${TARGET_DEVICE}2 

Let’s not add more write cycles onto the sdcard than necessary. Shrink the partition.

sudo resize2fs -M /dev/${TARGET_DEVICE}2 

LUKS encrypt the partition. Because these devices do not have AES intrinsics, use Adiantum (below). If you’re reading this from the future, check if an ASCON cipher is available.

Generate your password from a password manager application.

NOTE: Don’t use a 99 character password, because you will timeout during the splash screen on cold boot while trying to type it all in. Ask me how I know :man_facepalming:

Anyway, here’s wonderwall.

sudo cryptsetup-reencrypt --new --reduce-device-size=16M --type=luks2 -c xchacha20,aes-adiantum-plain64 -s 256 --hash sha3-256 --iter-time=8000 --pbkdf argon2id /dev/${TARGET_DEVICE}2 

Decrypt the LUKS partition and then re-expand the inner (ext4) partition.

sudo cryptsetup luksOpen /dev/${TARGET_DEVICE}2 rootfs 
sudo resize2fs /dev/mapper/rootfs 

Mount the root partition so that we can work inside of it.

sudo mkdir mnt 
sudo mount /dev/mapper/rootfs mnt 

Connect up the crypttab.

sudoedit mnt/etc/crypttab 

Add:

rootfs /dev/mmcblk0p2 none luks 

Connect up the fstab.

sudoedit mnt/etc/fstab 

Update such that the first line reads:

/dev/mapper/rootfs / ext4 defaults,noatime 0 0 

Sidebar: Partition Sizing

By default, ubuntu will expand the root partition to take up the entire disk. You may wish to disable auto-expand the root partition on bootup. Small disks make for small backup images.

Connect up to the cloud config.

sudoedit mnt/etc/cloud/cloud.cfg 

Remove growpart.

If you do this, next manually extend the partition now to your desired size.
Don’t skip expanding the partition; it avoids no-space-left-on-device-ing upon the first apt update && apt upgrade in 22.04.

I use 12GB below.

sudo umount /dev/mapper/rootfs 
sudo cryptsetup luksClose rootfs 

sudo parted /dev/sda resizepart 2 12GB 

sudo cryptsetup luksOpen /dev/${TARGET_DEVICE}2 rootfs 
sudo cryptsetup resize rootfs 

End of Sidebar: Partition Sizing

Mount the boot partition so that we can work inside of it.

sudo mkdir boot 
sudo mount /dev/${TARGET_DEVICE}1 boot 

Connect up the boot cmdline.

sudoedit boot/cmdline.txt 

Update

root=LABEL=writable

to

root=/dev/mapper/rootfs

and add cryptdevice=/dev/mmcblk0p2:sdcard to the end of line.

Your cmdline.txt should look akin to:

console=serial0,115200 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mapper/rootfs rootfstype=ext4 rootwait fixrtc quiet splash cryptdevice=/dev/mmcblk0p2:sdcard 

While you’re here, might as well enable USB data on the USB-C port.

Connect up the config.txt.

sudoedit boot/config.txt 

and place under [pi4]:

dtoverlay=dwc2,dr_mode=host 

Unmount all.

sudo umount /dev/${TARGET_DEVICE}1 
sudo umount /dev/mapper/rootfs 
sudo cryptsetup luksClose rootfs 
sync 

Shutdown, set aside your temporary OS sdcard if you used one, and install the newly encrypted sdcard into the RPI4.

Connect up monitor and keyboard if you haven’t already.

Now here comes some jank. We have to boot,

WAIT ON THE BLINKING CURSOR SEVERAL MINUTES FOR FAILOVER
WAIT ON THE BLINKING CURSOR SEVERAL MINUTES FOR FAILOVER
WAIT ON THE BLINKING CURSOR SEVERAL MINUTES FOR FAILOVER

, before getting dropped into an initramfs prompt.

Once you are booted into initramfs:

(initramfs) cryptsetup luksOpen /dev/mmcblk0p2 rootfs 
(initramfs) exit 

Login with user ubuntu password ubuntu. You’ll be forced to change the password now.
Update the initramfs so that we don’t hang at the blinking cursor anymore.

sudo update-initramfs -u 
sudo reboot 

You should now get the ubuntu boot password splash screen. Verify it works by giving it the LUKS password and observe the machine boot.

Remote Unlock

I recommend layering in dropbear for remote unlock, especially for headless machines.

Borrowing from Ubuntu Server 22.04 LTS with Remote LUKS Unlock - Migrating to Cockpit (Part I) ,

Bring the OS up to date and install dropbear.

sudo apt update 
sudo apt upgrade -y 
sudo apt install -y dropbear-initramfs  

Ignore this warning because we’re going to fix it next!

dropbear: WARNING: Invalid authorized_keys file, SSH login to initramfs won't work! 

Connect up to the dropbear configuration.

sudoedit /etc/dropbear/initramfs/dropbear.conf 

Uncomment and use:

DROPBEAR_OPTIONS="-I 300 -j -k -p 2222 -s"  

On your client machine,

ssh-keygen -t ed25519 -f ~/.ssh/unlock_dropbear 
cat ~/.ssh/unlock_dropbear.pub 

Place your public key on the RPI4:

echo "ssh-ed25519 AAAAC3...." | sudo tee /etc/dropbear/initramfs/authorized_keys 

Build the remote unlock key into the initramfs and reboot.

sudo update-initramfs -u 
sudo reboot 

Verify that your dropbear key works as well as unlocking the LUKS partition:

ssh -i ~/.ssh/unlock_dropbear -p2222 root@<IP_OF_RPI4> 

From inside the dropbear shell:

cryptroot-unlock 

Consider updating your client’s ssh config, too.

edit ~/.ssh/config

 
Host rpiname.dropbear 
    Hostname <IP_OF_RPI4> 
    User root
    Port 2222 
    IdentityFile ~/.ssh/unlock_dropbear 

If you happen to be using an X715 multi-voltage input hat like I am, use this updated variable-rate fan driver for ubuntu 22.04:

sudo apt install -y build-essential 
mkdir -p /tmp/fandriver 
pushd /tmp/fandriver 
git clone --depth 1 https://github.com/coricarson/X715.git 
pushd X715 
sudo make install 
popd 
popd 
rm -rf /tmp/fandriver 

Take a backup
If this is the third or fourth time you’re reading this guide, it probably means something inevitably went wrong. Happens all the time. I’ve been through this enough times that I ended up writing this post.

Take a backup image from e.g. your temporary OS booted on the rpi4.

lsblk 
 
read TARGET_DEVICE #e.g. "sda" 
export TARGET_DEVICE 

sudo fdisk -l /dev/${TARGET_DEVICE} 
Disk /dev/sda: 119.25 GiB, 128043712512 bytes, 250085376 sectors 
Disk model: Storage Device 
Units: sectors of 1 * 512 = 512 bytes 
Sector size (logical/physical): 512 bytes / 512 bytes 
I/O size (minimum/optimal): 512 bytes / 512 bytes 
Disklabel type: dos 
Disk identifier: 0x61768bc1 
  
Device     Boot  Start      End  Sectors  Size Id Type 
/dev/sda1  *      2048   526335   524288  256M  c W95 FAT32 (LBA) 
/dev/sda2       526336 23437500 22911165 10.9G 83 Linux 

Grab the block size (512 expected for sdcard) and the end of the second partition PLUS ONE: 23437501.

sudo dd if=/dev/${TARGET_DEVICE} of=rpi4_project.2023.02.14.img bs=512 count=23437501 status=progress 

# Optional: consider creating parity files to mitigate bitrot
sudo apt install -y par2 
par2 c -r15 ./rpi4_project.2023.02.14.img 

Zymkey 4i
This is a zymbit user forum is it not? Now, physically install your zymkey 4i: Quickstart - ZYMKEY4 | .

The setup package supports jammy! Thanks Zymbit!

curl -G https://s3.amazonaws.com/zk-sw-repo/install_zk_sw.sh | sudo bash 

Let the rpi4 reboot.
Unlock it manually at the terminal or via dropbear ssh, add a new key to the LUKS, and lock it with the zymkey 4i.

Borrowed from Ubuntu 20.04 and TPM2 encrypted system disk - Running Systems ,

sudo su -
pushd /root

cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 64 > root.key

num_times=4
while [ ${num_times} -gt 0 ] 
do 
    zklockifs root.key > /var/lib/zymbit/key.bin.lock 

    err=$? 
    if [ ${err} -eq 0 ] 
    then 
        break 
    else 
        num_times=$((num_times-1)) 
        sleep 0.2 
    fi 
done 

cryptsetup luksAddKey /dev/mmcblk0p2 root.key

rm root.key

popd

Now, add new hooks for initramfs generation. Borrowed from Zymbit’s own mk_encr_ext_rfs.sh script:

sudo su -

# Copy /var/lib/zymbit and all standalone zymkey utilities to initramfs
cat > /etc/initramfs-tools/hooks/zymkey_cryptfs_cfg <<'EOF'
#!/bin/sh

PREREQ=""

prereqs() {
     echo "$PREREQ"
}

case "$1" in
    prereqs)
        prereqs
        exit 0
        ;;
esac

. /usr/share/initramfs-tools/hook-functions

mkdir -p ${DESTDIR}/var/lib/zymbit
cp -prf /var/lib/zymbit/* ${DESTDIR}/var/lib/zymbit
copy_exec /sbin/zkunlockifs /sbin

EOF
chmod +x /etc/initramfs-tools/hooks/zymkey_cryptfs_cfg



# Bring the i2c drivers into initramfs
grep -q "^i2c-dev" /etc/initramfs-tools/modules || echo "i2c-dev" >> /etc/initramfs-tools/modules
grep -q "^i2c-bcm2835" /etc/initramfs-tools/modules || echo "i2c-bcm2835" >> /etc/initramfs-tools/modules
grep -q "^i2c-bcm2708" /etc/initramfs-tools/modules || echo "i2c-bcm2708" >> /etc/initramfs-tools/modules
grep -q "^lan78xx" /etc/initramfs-tools/modules || echo "lan78xx" >> /etc/initramfs-tools/modules

From Raspberry pi4 Ubuntu not booting - #10 by kontrasec , the stock keyscript has no fallbacks in case the zymbit is missing, damaged, or the OS was updated.

Run below to replace it:

cat > /lib/cryptsetup/scripts/zk_get_key <<'EOF'
#!/bin/sh

# Wait for network, but timeout if we can't see it.
num_times=30
while [ ${num_times} -gt 0 ]
do
    ls /sys/class/net/eth* 1>/dev/null 2>&1
    eth=$?
    ls /sys/class/net/enx* 1>/dev/null 2>&1
    enx=$?

    if [ ${eth} -ne 0 ] && [ ${enx} -ne 0 ]
    then
        num_times=$((num_times-1))
        sleep 0.1
    else
        break
fi
done

# Wait for zymbit, but timeout if we can't see it.
num_times=30
while [ ${num_times} -gt 0 ]
do
    if [ -d "/var/lib/zymbit" ]
    then
        break
    else
        num_times=$((num_times-1))
        sleep 0.1
    fi
done

if [ -e /var/lib/zymbit/zkenv.conf ]
then
    export $(cat /var/lib/zymbit/zkenv.conf)
fi

num_times=4
while [ ${num_times} -gt 0 ]
do
    /sbin/zkunlockifs /var/lib/zymbit/key.bin.lock

    err=$?
    if [ ${err} -eq 0 ]
    then
        exit
    else
        num_times=$((num_times-1))
        sleep 0.2
    fi
done

/lib/cryptsetup/askpass "Zymkey 4i did not release LUKS key. Please unlock rootfs with manual key:"

EOF
chmod +x /lib/cryptsetup/scripts/zk_get_key

Connect up to the crypttab.

sudoedit /etc/crypttab 

And add the keyscript= to the end:

rootfs /dev/mmcblk0p2 none luks,keyscript=/lib/cryptsetup/scripts/zk_get_key 

Now, rebuild the initramfs, reboot.

update-initramfs -u -k all 
reboot  

You should now observe your rpi4 operating in three different modes:

  1. With zymkey 4i attached, full disk encryption (FDE) with LUKS will auto-unlock on power on or reboot.
  2. Upon failure or removal of Zymkey 4i, remote unlock over dropbear SSH.
  3. Upon failure or removal of Zymkey 4i, local console unlock with monitor/keyboard.

Remember to take frequent backups.
Happy deployments!

Hi, thanks for the guide. Unfortunately, I must be doing something wrong in this dance of mounting/unmounting, reducing/increasing partitions because I keep getting the same error after rebooting and dropping in initramfs.

Is there an apparent point of failure that might result on that from your experience?

Regards,

Israel.

You are missing cryptsetup in your initramfs. I’ve noticed that the server image tends to include it, and the desktop image does not.

Boot into the unencrypted image, apt install cryptsetup and make sure initramfs is updated (update-initramfs), then reboot.