Hi Fred,
Thanks for your thoughtful question.
The attack vector you describe assumes that an attacker has free, undetected physical access to the SD card.
Zymkey is designed to counter this attack vector. It does so by providing both PHYSICAL and DIGITAL security features that work together (but independent of the host RPi) to detect physical tamper events and respond by disabling or destroying key materials (depending upon the policy you set).
The most important component of PHYSICAL security is the tamper detection circuits available on Zymkey 4i. When integrated with your enclosure they provide a secure envelope in which your RPi & SDCard are protected from physical attack. Learn more > Using Perimeter Detect.
If you need help in integrating tamper detection into your physical solution, then this Protokit is a good starting point > ProtoKit 4S
Hopefully this completes the missing piece in your understanding (physical security). Let us know if you have more questions.