Encrypting Your Root File System on RASPBERRY PI - using LUKS & dm-crypt

ZYMKEY4 Encrypting File System
41

@mark Sorry for the late response and thanks for describing your use case. It’s an interesting and logical one, to be sure.

I think it would be completely possible to have Zymkey manage LUKS keys for two separate volumes. The basic flow would go something like this:

  1. Kernel boots with initramfs
  2. initramfs mounts and hands over to encrypted rootfs
  3. /etc/fstab and /etc/crypttab describe other encrypted volumes. The rootfs key could be used or unique keys could be generated for each separate volume.

I would start with encrypting the rootfs with either the SD card conversion or the external USB migration script. Once you have that set up and booting, then you could encrypt your secondary external drive referring the external USB script as a guideline, then soft links could be set up on the encrypted rootfs environment to point to the important files or directories that live on the secondary drive.

42

It may well be that my understanding is flawed, but this thing bugs me still. Given the 4i Lite without tamper detection, how would you mitigate a scenario where the bad actor modifies the initramfs to get a login tty and then just stays in the initramfs indefinitely without mounting the encrypted rootfs as root?

Essentially they’d have all the required software in the inherently unprotected initramfs to use Zymbit’s API to decrypt the LUKS key and then they could just mount and browse the rootfs as a simple mount, if I’m not mistaken.

Does the Zymkey somehow verify the initramfs image the mk_encr_sd_rfs.sh script creates to prevent this? Or does the Zymkey present the decrypted key to LUKS or other subsystem in a way that the decrypted key cannot be intercepted (e.g. from a memory dump)?

43

Hi @hlev80,
You are correct in your understanding as it pertains to an encrypted root file system on SD card with the 4i Lite. As I mentioned in a previous post, we currently don’t do anything to protect/validate the /boot partition, so a bad actor could exercise Zymkey to get at the LUKS key. I should mention a couple of things here:

  1. We are currently working on strategies that validate the critical boot components. These are in the design and development stages.
  2. If one was to use the external migration script instead of the SD card conversion script to migrate the root file system to an external USB mass storage device, an extra level of security could be achieved by permanently write protecting the SD card (https://github.com/BertoldVdb/sdtool). Since the Zymkey binds itself to the host computer and the SD card ids and the SD card would be locked, the system would be a little more secure because the bad actor could not simply rebuild the initrd with his own malware onto a new SD card. The only real drawback to this approach is that the kernel could not be upgraded. However, in a production environment, I think most people would agree that major components like the kernel should be locked down for the life of deployment for the sake of stability.

Having said the above, if one has physical access to a device, a lot more attack vectors become exposed no matter how good the cryptographic design is. That’s why we strongly recommend tamper detection as part of a robust IoT deployment strategy. With the tamper detection feature configured for “self destruct” mode, the Zymkey will turn itself into a brick after the detection of a tamper event.

44

Hi Guys,
Im interested with Zymkey to protect my Raspberry pi.
But I got questions:
Actually my project will be temperature and pressure monitoring.
I will be using 4 pcs ADS1115 ADC that uses the I2C
with the following address:
0x48 (1001000)
0x49 (1001001)
0x4A (1001010)
0x4B (1001011)

This will not interrupt or zymkey and my devices will work?

  1. If I LUKS encrypt and apply zymkey to /home/pi folder with my application is also located that will run the programs for my temp and press monitoring, will the program will still work even the system hung and reboot itself with an external watchdog timer?
    The program is run on startup using the crontab as a task.

looking forward for your reply soon for me to decide to buy Zymkey.

Regards,

45

Hi @azkikr,

Thanks for your interest in Zymkey. The Zymkey 4i default i2c address is located at 0x30, so there should be no conflict with the devices you listed.

Regarding your second question, please keep in mind that this community post details how to encrypt the entire root file system. If your system hangs or is reset by a watchdog timer, there is always a small possibility that some files on the root file system may become corrupt. This is true with or without encryption. In order to avoid these kinds of issues, it is always a good idea to shut down gracefully whenever possible. Having said that, the ext4 file system is probably the most robust file system in existence today with regards to recovery. This is due in part to good journaling design.

46

Hi @Scott_of_Zymbit,

  1. Thanks for the reply and noted about the I2C address.s

  2. Understand about this.
    My application I am running is a simple python script that gets the monitoring data every 5 minutes. The program is not looping instead it is run by a task every 5 minutes using crontab. Hence corrupted file maybe not an issue.
    The raspberry pi is on remote location which not easily accessible by people to reset in case the raspberry pi hangs. So I put a watchdog timer to reset it automatically.
    My question is if the raspberry pi was reset is the raspberry will boot automatically and run with its scheduled task to run my python program?
    Or does it need a human intervention to reset the zymkey in order for the Pi to boot itself?

If it can boot automatically is there a chance that zymkey will prevent it on booting up the Pi?

47

If the RPi reboots as a result of the watchdog timer, the Zymkey will reboot itself as well because it will no longer be receiving heartbeats from the Zymkey interface daemon.

48

Is this the solution for colocation of a raspberrypi in a data center?

I tried encrypting the root with Luks on a Odroid C2 that was in a data center. Worked great until a firmware update borked it. The way I had it working was to secure shell into initramfs and enter password. Than ssh again after system comes up.

So what happens if someone physically steals my raspi or connects to serial port? Is my data safe using your zymbit module? Can I set it up to still be asked for a passphrase after reboot via remote ssh session? Your system sounds like what I am looking for.

49

Our root file system conversion/migration scripts don’t currently configure a backup password. However, this can certainly be accomplished with a little work. Or, as you suggested, you could certainly set up something like dropbear in the initramfs.

Another idea that you might be interested in that we are working on is the concept of one or more backup Zymkeys. In one use case, you could safely store a backup key in a safe place and, when the Zymkey was compromised via tamper detection, the backup key could be installed so that the data could be recovered.

To answer your larger question, we believe that if you configure a secure enclave with Zymkey tamper detection, Zymkey could certainly work for you.

50

Hey @Scott_of_Zymbit

So I just got the 4i and I want to secure my RPi using the luks encryption and allow for remote access using SSH and I got some questions for you.

  1. after installing the i4 (to the point where it blinks every 3 seconds), all I need to do to encrypt the entire sd card and save the master key to the i4 is run the command written in option 1 (with the relevent parameters)?
  2. after the sd is encrypted, how do I unlock it? will it open up just when the i4 will be connected upon reboot? is there a password I need to set and use upon login?
  3. using option 1 I need an external drive to be connected to the RPi, should it be empty? or with a raspi installed? and do I need to change configurations with the RPi so it will boot with the usb drive instead of the SD?
  4. after having the encrypted system, is it possible in any way to remotly connect to it? transfer files (from/to)?
  5. which distributions works best with the i4?

Sorry for all the questions I’m kinda new to linux/RPi.

Thanks, Ido.

51
  1. Run the SD card conversion script. BTW, this only encrypts the root file system, not the boot partition.
  2. The SD card conversion script is meant to boot an encrypted root file system in unattended mode, which is a use common use case for an IoT device. So, no password is necessary. You can prove this by popping the SD card with an encrypted rfs into a Linux PC and trying to mount the rfs. You will see that it will ask for a passphrase and if you try to read the sectors on the disk, they will be indecipherable.
  3. It would be better if the external drive was empty, but the entire device will be wiped out and reconfigured anyway.
  4. The encrypted root file system will behave just like it did before the encryption phase. As described in 2., it won’t be able to be read outside of the parent Pi.
  5. We support Raspbian Jessie and Stretch today. They both work well, but Stretch is more recent with a lot of improvements, so I’d concentrate on that distro.

No problems about the questions. That’s what we’re here for :slight_smile:

52

Thanks scott,

I got the encryption and zymkey working.
To test the encryption I tried to load the same SD on a different pi and after that failed I inserted the Zymkey to the new pi which resulted in a working pi. I just wanted to make sure that the reason it worked is becuase the Zymkey cut-2-lock tab was still intact so the module just paired with the new pi, which means that if I’ll cut it the system will be unlocked only if I’ll use the paired pi + the zymkey, right?

53

That’s correct. Once you cut the lock tab, the Zymkey cannot be moved to a different Pi + SD card.

54

Do subsequent passes after the first reboot run in the background? I got through the first pass fine, then when my pi rebooted and just sat on the desktop for a bit and I didn’t see anything happen so I assumed I had to execute the curl command again. I later read in a reply that the subsequent passes should initiate themselves…currently my pi boots up fine, zymkey LED is off.

55

@Scott_of_Zymbit

Quick question - Working to get this working with Kali Linux, a derivative of debian-testing, and running into a bit of an issue.

Using slight modifications to the getting started process I am able to get Kali working just fine with the Zymkey, and the testing apps work great. When I run the SD card conversion, it completes correctly however in the end when it boots up off of the SD card it falls to the initramfs shell. Before that, the /scripts/local-block script is called multiple times and there is a couple errors about “ifconfig: no devices to configure”. There is also a call out of “ALERT! /dev/mapper/cryptrfs does not exist”.

And during this process, the Zymkey light rapidly blinks.

Looking at phase one output, there is only one real error that comes forward:

Created symlink /etc/systemd/system/multi-user.target.wants/rsync.service → /lib/systemd/system/rsync.service.
Setting up zksaapps (1.0-8) ...
Processing triggers for systemd (238-5) ...
Stopping zkifc...done.
mount: /mnt/tmproot: wrong fs type, bad option, bad superblock on /dev/sda1, missing codepage or helper program, or other error.
Mounting failed. Installing crypto installer on /dev/sda.
Installing necessary packages...
umount: /dev/sda1: not mounted.
done.
Formatting USB mass media on /dev/sda...1+0 records in
1+0 records out
512 bytes copied, 0.010963 s, 46.7 kB/s
mke2fs 1.44.2 (14-May-2018)
Making a tarball of original root file system image...tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
done.
Created symlink /etc/systemd/system/multi-user.target.wants/cfg_SD_crfs.service → /etc/systemd/system/cfg_SD_crfs.service.
Removed /etc/systemd/system/multi-user.target.wants/zkifc.service.
Removed /etc/systemd/system/multi-user.target.wants/zkbootrtc.service.
  5,199,998,326  99%   11.82MB/s    0:06:59 (xfr#173238, to-chk=0/194775)
sed: -e expression #1, char 4: extra characters after command
done.
root file sys conversion phase 1 complete.
Rebooting to installer partition to start phase 2...

Stage 2 seems to run, as the device comes up, I can SSH into it and validate its “doing stuff” and it is clearly running off of the USB device. If I pop the SD card into another system I can see that yes, the 2nd partition is ‘crypto_LUKS’, and the cmdline.txt and config.txt seems to be fine.

root@kali:/mnt# cat cmdline.txt
dwc_otg.fiq_fix_enable=2 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1  rootwait rootflags=noload net.ifnames=0  root=/dev/mapper/cryptrfs cryptdevice=/dev/mmcblk0p2:cryptrfs
root@kali:/mnt# tail config.txt
##
##     In case you want to connect it to an external device
##
##     Not available on Model A/B boards.
##
##     Default 35.
##
#dtparam=pwr_led_gpio=35
dtparam=i2c_arm=on
initramfs initrd.img followkernel

Looking at the initrd.img I do see:

lsinitramfs /mnt/initrd.img | grep crypt
usr/lib/arm-linux-gnueabihf/libcryptsetup.so.12.2.0
usr/lib/arm-linux-gnueabihf/libgcrypt.so.20.2.3
usr/lib/arm-linux-gnueabihf/libcryptsetup.so.12
usr/lib/arm-linux-gnueabihf/libgcrypt.so.20
lib/arm-linux-gnueabihf/libcryptsetup.so.12
lib/arm-linux-gnueabihf/libgcrypt.so.20
lib/modules/4.9.80-Re4son-v7+/kernel/crypto
lib/modules/4.9.80-Re4son-v7+/kernel/crypto/xor.ko
lib/modules/4.9.80-Re4son-v7+/kernel/crypto/cryptd.ko
lib/modules/4.9.80-Re4son-v7+/kernel/crypto/ablk_helper.ko
lib/modules/4.9.80-Re4son-v7+/kernel/drivers/block/cryptoloop.ko
lib/modules/4.9.80-Re4son-v7+/kernel/drivers/md/dm-crypt.ko
lib/modules/4.9.80-Re4son-v7+/kernel/arch/arm/crypto
lib/modules/4.9.80-Re4son-v7+/kernel/arch/arm/crypto/aes-arm.ko
lib/modules/4.9.80-Re4son-v7+/kernel/arch/arm/crypto/aes-arm-bs.ko
lib/cryptsetup
lib/cryptsetup/askpass
scripts/local-top/cryptopensc
scripts/local-top/cryptroot
scripts/local-bottom/cryptopensc
scripts/local-block/cryptroot
bin/cryptroot-unlock
sbin/cryptsetup

Before I started to bash my head into this too much, I thought I would just ask here and see if you have encountered this before and I might be able to save some time on getting this going.

Any direction you might provide would be great. Thanks.

56

I get the same as @smbrandnojr: pass 2 does not start after reboot. When I start it manually, I get:

...
Stopping zkifc...done.
cp: cannot stat '/var/lib/zymbit/': No such file or directory
cp: '/etc/fstab' and '/mnt/tmproot/etc/fstab' are the same file
sed: -e expression #1, char 4: extra characters after command
done.
root file sys conversion phase 1 complete.
Rebooting to installer partition to start phase 2...

But after reboot pass 2 does not start. The USB stick is still the root filesystem. Zymkey light is off.
I used 2018-06-27-raspbian-stretch-lite.img. I disabled the automatic filesystem resize before the first boot (but that should not matter, should it?).

EDIT: after the reboot after pass 1, the Zymkey LED blinks at 5Hz (like when it was not bound yet), only after manual restart of the script it turns off (probably because of Stopping zkifc...)

57

I am also having a similar problem as @smbrandnojr. The difference is that during phase 2, the blue led is flashing at ~2hz (like when you initially put the zymbit on) as opposed to smbrandnoir’s which was just off.

Phase 1 took 5 minutes (I did have to do a wipefs /dev/sda* like the install instructions told me to do)
Phase 2 has been running > 1 hr so far. It is an 8GB file system if that helps.

58

It may be worth verifying you are on a clean power supply and also maybe try and limit any peripheral devices (USB, HDMI) that you may have plugged in. I got my issue resolved, but it seemed more related to possibly finicky power sensitivities (I am using a canakit power adapter for the pi, and it was plugged into an uninterruptible power supply and I also had a keyboard usb dongle plugged in along with an hdmi monitor)

Now that I am thinking about it, the HDMI monitor is on a different power supply and I’m wondering ultimately that was the cause. Either way, I unplugged the HDMI monitor after starting the first phase and then went and did other things for a bit. When I came back, the module LED was blinking like it was in operation so I plugged stuff back in. When ever I rebooted my pi with the monitor plugged in, it would fail to boot probably 4 out of 5 attempts. Monitor unplugged, it reboots fine everytime and I ssh in to it.

59

Some more info:

Pass 2 does not appear to start. Despite the systemctl enable cfg_SD_crfs it does not start. When I manually run (as root) /usr/local/bin/cfg_SD_crfs.sh, pass 2 completes and the pi reboots with the SD card as encrypted root filesystem (yea :grinning:). But the zymkey loses binding. It takes another reboot to get the zymkey to stay bound (finally :grinning:).

After seven additional reboots five are ok (system boots, zymkey bound) and two result in a kernel panic. I have nothing connected except a USB stick and a HDMI monitor. The power supply is 2A switched (Chinese made BX-502000, used to be official RPi power supply). There are no signs of under-voltage (no lightning bolt on console, no red LED stutter). I’ll try a different power supply tomorrow.

I really like the zymkey and “we” (the company I work for) would like to deploy hundreds of devices in the field. But, as devices will be spread out and not easily accessible, we need reliability. A trip to the device upon kernel panic is not an option.

60

@scriptmelvin
Good to hear you are making progress - your intermittent reliability issues have all the symptoms of poor power quality. If the zymkey was defective, or code not working, then the zymkey simply would not work.

The 2A supply is under the recommended 2.5A, and depending on what you have connected to your system (display, keyboard, other… ), you may need 4A.

We are always open minded to feedback, but I think there is a good chance the right power supply will solve your problems. Let us know it works with upgraded power .

(As a side note, which Zymkey are you using - standard or lite - and which host Pi?)