Encrypting Your Root File System on RASPBERRY PI - using LUKS & dm-crypt

Phil,

Have a source for 4A (or higher) microusb adapters?

@Phil_of_Zymbit, thx for your answer and suggestions. I am using a zymkey 4i (not the lite) and a RPi 3.

The pass 2 service not starting does not seem to be power supply related… But I’ll report back here after power supply upgrade.

I bought some stuff and tried again.

  • stuff used: new RPi 3, new SD card 8GB class 10, new USB stick 16GB, new power source 5.1V 5A, attached to 5V/GND of RPi GPIO header, only ethernet and HDMI attached to RPi
  • write 2018-06-27-raspbian-stretch-lite.img to SD card
  • write empty file ssh to /boot
  • boot RPi, RPi does its resize thing and reboots
  • attach scope to 5V/GND of RPi GPIO header, set to trigger and hold on falling edge at 4.76V
  • login with ssh, raspi-config, set locale, timezone, enable i2c, disable i2c, enable i2c
  • wipefs -a /dev/sda (just to be sure)
  • reboot RPi
  • install Zymkey sw: curl -G https://s3.amazonaws.com/zk-sw-repo/install_zk_sw.sh | sudo bash
  • RPi reboots, Zymkey becomes bound
  • start LUKS conversion: curl -G https://s3.amazonaws.com/zk-sw-repo/mk_encr_sd_rfs.sh | sudo bash
  • during apt-get update (I presume):

    Err: http://archive.raspberrypi.org/debian stretch Release
    503 Service Unavailable [IP: 2a00:1098:0:80:1000:13:0:7 80]

    E: The repository ‘http://archive.raspberrypi.org/debian stretch Release’ does no longer have a Release file.
  • tar sometimes complains about file changed as we read it / File removed before we read it

I did this twice, first time I did apt-get update and apt-get upgrade and reboot before installing any Zymkey sw, phase 2 then started automatically. After phase 2 the kernel panicked. Power cycled twice, kernel panicked twice more. The second time I did NOT update/upgrade, and phase 2 did NOT autostart, and I started it manually. Now the system started normally and the Zymkey got bound. Yeay.

Then I did 10 reboots. On the 4th, 6th, 7th, 9th and 10th reboot: kernel panic. Power cycled to get to 11th boot and working system and bound Zymkey.

The scope never triggered.

Does anybody have any idea what’s going on?

Just curious if you unplug your HDMI and power cycle, if the rPi boots and zymkey binds consistently.

@scriptmelvin if you can send us a private message of when when you bought your Zymkeys and the order number we will take a closer look at our end. support@zymbit.com

@smbrandnojr: brilliant! I did 10 reboots and 3 shutdown -h nows followed by power cycle without HDMI connected: working system all times! Only on the 8th reboot the Zymkey was unbound… This looks reliable enough together with the RPi watchdog.

The HDMI cable is actually a HDMI to DVI cable connected to an old monitor I had lying around.

Tomorrow I’ll try the whole process again, with and without apt-get update/upgrade (and without HDMI :wink:).

@Phil_of_Zymbit: order number is in the office, will do on Monday.

Thx everyone!

EDIT: I wonder if I might have set the scope incorrectly…

Hi @scriptmelvin,

I shot you a message, but was wondering if I could get some info on rPi watchdog…I’m having issues implenting it on my 3 B+

Mike

Did you follow a specific guide for the rPi watchdog? The few I have followed, I get error on sudo modprobe bcm2708_wdog…bcm2708 not found

That’s correct. The module got renamed to bcm2835_wdt. See this. Haven’t tried myself yet, my first goal is a reliable Zymkey setup :slight_smile:

@smbrandnojr: this seems to work for me. To test you can deliberately cause a kernel panic with:

sudo sh -c 'echo c > /proc/sysrq-trigger'

Don’t want to test myself right now, I’m not physically near the RPi.

It’s a power issue after all. Today during phase 2 the red LED started flashing wildly. The header extender (to get longer pins) looks flimsy (pins too flat/thin). Will apply proper power and retry. Thx all and sorry for the confusion.

EDIT: all working well now. The only remaining issue seems to be: if I don’t do an apt-get update and apt-get upgrade (before zk sw installation) phase 2 autostarts. If I do an apt-get update and apt-get upgrade phase 2 does not autostart.

Pass 2 still sometimes fails. I now have the new official RPi USB power brick (5.1V, 2.5A) connected, an ethernet cable, a USB stick and nothing else. More info:

pi@raspberrypi:~ $ sudo journalctl -u cfg_SD_crfs.service 
-- Logs begin at Thu 2016-11-03 18:16:42 CET, end at Thu 2018-07-26 11:52:40 CEST. --
Jul 26 11:18:06 raspberrypi systemd[1]: Started First time boot encrypted filesystem cfg service.
Jul 26 11:18:12 raspberrypi cfg_SD_crfs.sh[304]: Creating LUKS key...Session ACK not received
Jul 26 11:18:12 raspberrypi cfg_SD_crfs.sh[304]: led_msg failed
Jul 26 11:18:12 raspberrypi cfg_SD_crfs.sh[304]: Unexpected response received. sessionMsgID = 0100, rx msgid = 39e2, expected_msg_id = 0007
Jul 26 11:18:12 raspberrypi cfg_SD_crfs.sh[304]: led_msg failed
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: Session ACK not received
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: led_msg failed
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: Unexpected response received. sessionMsgID = 0100, rx msgid = 8d9d, expected_msg_id = 0007
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: led_msg failed
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: done.
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: Formatting crypto file system on /dev/mmcblk0p2...Command failed with code 32.
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: Device /dev/mmcblk0p2 is not a valid LUKS device.
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: done.
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: Creating ext4 partition on /dev/mmcblk0p2...mke2fs 1.43.4 (31-Jan-2017)
Jul 26 11:18:20 raspberrypi cfg_SD_crfs.sh[304]: The file /dev/mapper/cryptrfs does not exist and no size was specified.
Jul 26 11:18:20 raspberrypi systemd[1]: cfg_SD_crfs.service: Main process exited, code=exited, status=1/FAILURE
Jul 26 11:18:20 raspberrypi systemd[1]: cfg_SD_crfs.service: Unit entered failed state.
Jul 26 11:18:20 raspberrypi systemd[1]: cfg_SD_crfs.service: Failed with result 'exit-code'.

Update: I got it working with a base image of stretch. The zymkey locks, encrypts the root file system, and flashes every 3 seconds verifying it’s locked to the pi again.

Trying with the custom image (~9GB), it took ~5 hours to finish phase 1. I did get a couple tar errors of “a file changed while looking at it” and “ignoring socket”.

Phase 2 started, and I didn’t have keyboard or HDMI attached as recommended by another user. I waited the weekend, and it didn’t lock.

Restarting it with a monitor and keyboard, it ran for a bit and restarted on its own. I could log in, but everything was slow due to bzip2 taking up 10-100% of the CPU. The Zymbit also isn’t locked (flashing every 3 seconds), but the SD card is encrypted when I slap it in my laptop.

Does this mean phase 2 didn’t start until I restarted it, and now it’s going through it? I didn’t run sudo apt-get update like scriptmelvin mentioned, but the custom image I flashed most certainly has run that command in the past.

@Phil_of_Zymbit
Would it be possible to change the script to encrypt an additional partition with a shared, known key? This is similar to the question https://community.zymbit.com/t/same-encryption-key-for-2-zymkeys/115/3.

[Update 2]
Problem:
I’m not able to get the custom pi image to lock during Phase 2.

Possible reasons:
During phase 1 on ssh-ed terminal:


Partition #1 contains a vfat signature.
mke2fs 1.43.4 (31-Jan-2017)
Making a tarball of original root file system image…tar: Removing leading `/’ from member names

tar: /mnt/tmproot: file changed as we read it
tar: Removing leading `/’ from hard link targets
tar: /tmp/mongodb-27017.sock: socket ignored
done.
Created symlink /etc/systemd/system/multi-user.target.wants/cfg_SD_crfs.service → /etc/systemd/system/cfg_SD_crfs.service.
Removed /etc/systemd/system/multi-user.target.wants/zkifc.service.
Removed /etc/systemd/system/multi-user.target.wants/zkbootrtc.service.

On Pi’s side during Phase 1:

[12864.803985] EXT4-fs error (device sda1): ext4_validate_block_bitmap:385: comm rsync: bg 96: bad block bitmap checksum
[13651.681592] EXT4-fs error (device sda1): ext4_validate_block_bitmap:385: comm kworker/u8:4: bg 50: bad block bitmap checksum
[13656.704721] EXT4-fs (sda1): Delayed block allocation failed for inode 1054501 at logical offset 0 with max blocks 1 with error 74
[13656.709004] EXT4-fs (sda1): This should not happen !! Data will be lost
[13656.709004]

During Phase 2 on Pi’s side:

[ TIME ] Timed out waiting for device dev-disk-by\x2dpartuuid-7358e674\x2d01.device.
[DEPEND] Dependency failed for /boot.
[DEPEND] Dependency failed for Local File System.
[DEPEND] Dependency failed for File System Check on /dev/disk/by-partuuid/7358e674-01.

[264.624420] EXT4-fs error (device sda1): ext4_mb_generate_buddy:756: group 144, block bitmap [maybe another word here] clusters

Note: Any additional restart boots me in emergency mode or “Kernel panic -not syncing”.

Probably not the problem:

  1. Power: “under voltage detected” errors have occurred before, but a new power supply has eliminated the error in the past 6 hours it’s been running.
  2. USB un-formatted: “Could not mount” and other related errors occurred when using a USB from a failed zymbit attempt, and suggestions to use wipefs when the usb had been previously used as a Linux boot-from-usb. Both errors have been eliminated by cleaning & reformatting with Windows terminal.

Additional Hardware Information:

  • Raspberry P* i 3 B+ w/ Stretch
  • Running scripts through ssh-terminal from Ubuntu 18,04
  • Pi Powered by a 6 cell LiPo battery through a pdb
  • Wifi dongle attached to pi advertising wifi
  • Pi connected to monitor using PIP with an HDMI cable
  • Zymbit 4i (not lite)

Previous Attempts:

  • I’ve successfully implemented this twice on a base Stretch image
  • I’ve tried a full run through 4 times with the custom Stretch image. Power and mounting issues have occurred on the first two, but I don’t know how to continue from here.

I am testing now with a USB-bootable rPi3B+ followiyng these instructions (working): https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md

Wonder if there is a script tested from Zymkey for this case.
/dev/sda1 mounts boot and /dev/sda2 mounts /.

Maybe I could use SD-encryption method and then moving to USB as I have done with a non-encrypted SD-card.
Reason is I expect to have a much more robust device if using USB as storage than SDcards. Is this non-sense?

Why doesn’t phase2 start?
It does automatically?
i want to destroy everything

Hi @Iker,

We have a script which can migrate your root file system to an external USB device instead of converting the SD card. See “Option 2 - Migrate existing SD card to external LUKS storage device”.

This is a good strategy for robustness if you intend to use a hard drive or SSD.

Hi @pico2183,

Stage 2 starts automatically. It may take a long time depending on:

  1. The size of your existing root file system.
  2. The speed of your external device. We recommend using an SSD connected via SATA-USB converter.
  3. The speed of your SD card. Some SD cards are extremely slow on write even though they are rated as class 10.