Aws-iot-device-client Integration

chill · July 1, 2024, 7:02am

Hi Team,
We are using Zymbit SEN for our project. I noticed that zymbit supports PKCS11, and the AWS IoT device client also supports secure elements. I attempted to use the AWS IoT device client to log in with zymbit’s private key.

I referred to this tutorial: AWS IoT Device Client Secure Element README.

When executing the following method:

$ ./aws-iot-device-client --enable-secure-element [true|false] --pkcs11-lib [your/path/to/pkcs#11/library] --secure-element-pin [User PIN of PKCS#11 token] --secure-element-key-label [key-label] --secure-element-slot-id [token-slot-id] --secure-element-token-label [token-label]

I encountered the following error:

[DEBUG] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - Loading PKCS#11. file:'/tmp/lib/libzk_pkcs11.so' C_Initialize:yes
[INFO] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50: PKCS#11 loaded. file:'/tmp/lib/libzk_pkcs11.so' cryptokiVersion:2.40 manufacturerID:'SoftHSM' flags:0x6E656D656C706D49 libraryDescription:'tation of PKCS11        ' libraryVersion:0.0 C_Initialize:CKR_OK
[TRACE] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50: Found 2 slots with tokens. Picking one...
[TRACE] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50: Ignoring PKCS#11 token because slot 0 doesn't match 4747930165
[DEBUG] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50: Selected PKCS#11 token. slot:4747930165 label:'Chill' manufacturerID:'SoftHSM project' model:'SoftHSM v2' serialNumber:'6ae720ee9affaa35' flags:0x0000042D sessionCount:1099511627775/4294967295 rwSessionCount:18446744073709551615/18446744069414584324 freePublicMemory:0/3472333810982467120 freePrivateMemory:0/0 hardwareVersion:0.0 firmwareVersion:0.0
[DEBUG] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50 session=1: Session opened on slot 4747930165
[DEBUG] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50 session=1: User logged in
[ERROR] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50 session=1: Failed to find private key on PKCS#11 token which matches search criteria
[DEBUG] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50 session=1: Session closed
[ERROR] [2024-06-27T06:01:45Z] [0000007fb666dd40] [mqtt-client] - id=0x7fb666c850: Error initializing TLS context from PKCS11 options
[DEBUG] [2024-06-27T06:01:45Z] [0000007fb666dd40] [pkcs11] - id=0x7fa0000d50: Unloading PKCS#11. C_Finalize:omit

I checked, and the private key should exist.

p11tool --provider=/tmp/lib/libzk_pkcs11.so --login --list-all "pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=6ae720ee9affaa35;token=Chill" --debug 9
Setting log level to 9
|<2>| p11: Initializing module: /tmp/lib/libzk_pkcs11.so
|<2>| p11: No login requested.
Token 'Chill' with URL 'pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=6ae720ee9affaa35;token=Chill' requires user PIN
Enter PIN:
|<2>| p11: Login result = ok (0)
|<3>| ASSERT: ../../lib/pkcs11.c[pkcs11_import_object]:2173
|<2>| unknown pkcs11 object class 3
|<3>| ASSERT: ../../lib/pkcs11.c[pkcs11_import_object]:2145
|<3>| ASSERT: ../../lib/pkcs11.c[pkcs11_import_object]:2159
|<3>| ASSERT: ../../lib/pkcs11.c[pkcs11_import_object]:2168
|<3>| ASSERT: ../../lib/pkcs11.c[find_multi_objs_cb]:3360
|<3>| ASSERT: ../../lib/pkcs11.c[find_multi_objs_cb]:3138
Object 0:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=6ae720ee9affaa35;token=Chill;id=%00%00;object=iotkey
        Type: Unknown
        Label: iotkey
        Flags: CKA_PRIVATE; CKA_SENSITIVE;
        ID: 00:00

chill · July 2, 2024, 8:42am

By executing the aws-iot-device-client with root permission, we are able to load the private key from PKCS11 slot.
However, we are facing a new error:
Error log found in aws-iot-device-client.log

2024-07-02T08:34:13.520Z [ERROR] {SharedCrtResourceManager.cpp}: MQTT Connection failed with error: aws-c-io: AWS_IO_TLS_ERROR_NEGOTIATION_FAILURE, TLS (SSL) negotiation failed
2024-07-02T08:34:13.520Z [ERROR] {SharedCrtResourceManager.cpp}: Failed to establish shared MQTT connection, but will attempt retry...

Error log found in sdk.log

[WARN] [2024-07-02T08:30:30Z] [0000007f89f00d40] [tls-handler] - id=0x7f80013c20: negotiation failed with error TLS alert received (Error encountered in /usr/local/share/zymkey/aws/aws-iot-device-client/build/aws-iot-device-sdk-cpp-v2-src/crt/aws-crt-cpp/crt/s2n/tls/s2n_alerts.c:237)
[DEBUG] [2024-07-02T08:30:30Z] [0000007f89f00d40] [tls-handler] - id=0x7f80013c20: Alert code 70
[DEBUG] [2024-07-02T08:30:30Z] [0000007f89f00d40] [channel-bootstrap] - id=0x55a6541cb0: tls negotiation result 1029 on channel 0x7f80002890

The key was imported into pkcs11 in this way.

sudo zk_pkcs11-util --init-token --slot 0 --label "zymkey" --so-pin 1234 --pin 1234
The token has been initialized and is reassigned to slot 1656088301
sudo zk_pkcs11-util --use-zkslot 0 --slot 1656088301 --label iotkey --id 0000 --pin 1234

The csr and certificate was generated by following the tutorial AWS IoT - TLS Client Certificate Authentication |

openssl req -key nonzymkey.key -new -out zymkey.csr -engine zymkey_ssl -keyform e -subj "/C=US/ST=California/L=Santa Barbara/O=Zymbit/OU=Zymkey/CN=rpi.edge.zymbit.com"

then I signed a certificate from aws iot core console by uploading the csr,
I have also add the secure-element code into config:

  "secure-element": {
                "enabled": true,
                "pkcs11-lib": "../lib/libzk_pkcs11.so",
                "secure-element-pin": "1234",
                "secure-element-key-label": "iotkey",
                "secure-element-slot-id": 1656088301,
                "secure-element-token-label": "zymkey"
  }

The private is found correctly:

[INFO] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [pkcs11] - id=0x7f84000c70: PKCS#11 loaded. file:'../lib/libzk_pkcs11.so' cryptokiVersion:2.40 manufacturerID:'SoftHSM' flags:0x00000000 libraryDescription:'Implementation of PKCS11' libraryVersion:2.5 C_Initialize:CKR_OK
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [pkcs11] - id=0x7f84000c70: Found 3 slots with tokens. Picking one...
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [pkcs11] - id=0x7f84000c70: Ignoring PKCS#11 token because slot 1153640148 doesn't match 1656088301
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [pkcs11] - id=0x7f84000c70: Ignoring PKCS#11 token because slot 2 doesn't match 1656088301
[DEBUG] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [pkcs11] - id=0x7f84000c70: Selected PKCS#11 token. slot:1656088301 label:'zymkey' manufacturerID:'SoftHSM project' model:'SoftHSM v2' serialNumber:'1f8448ca62b5e6ed' flags:0x0000042D sessionCount:18446744073709551615/0 rwSessionCount:18446744073709551615/0 freePublicMemory:18446744073709551615/18446744073709551615 freePrivateMemory:18446744073709551615/18446744073709551615 hardwareVersion:2.5 firmwareVersion:2.5
[DEBUG] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [pkcs11] - id=0x7f84000c70 session=1: Session opened on slot 1656088301
[DEBUG] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [pkcs11] - id=0x7f84000c70 session=1: User logged in
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [pkcs11] - id=0x7f84000c70 session=1: Found private key. type=CKK_EC
[DEBUG] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [tls-handler] - ctx: Setting ALPN list x-amzn-mqtt-ca
[DEBUG] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Creating new mqtt 311 connection
[DEBUG] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-topic-tree] - tree=0x7f84030ec8: Creating new topic tree
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Setting connection success and failure handlers
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Setting connection interrupted and resumed handlers
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Setting connection closed handler
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Setting connection termination handler
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Setting username and password
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Setting reconnect timeouts min: 15 max: 240
[TRACE] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Opening connection
[DEBUG] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: Begin connecting process, switch state to CONNECTING.
[INFO] [2024-07-02T08:34:13Z] [0000007f9abf0d40] [mqtt-client] - id=0x7f84030bb0: using ping timeout of 3000000000 ns

The endpoint of aws IoT client is reachable by ping, but finally I got the error above.
Any suggestion will be welcome.

grfo · July 2, 2024, 9:19am

I don’t know the aws-iot-device-client, but I got the “AWS_IO_TLS_ERROR_NEGOTIATION_FAILURE: TLS (SSL) negotiation failed” error with the awsiotsdk because I was using the wrong root CA. Could this be your problem also?

chill · July 2, 2024, 9:24am

That’s possible, let me check.

davidgs_zymbit1 · July 2, 2024, 1:03pm

I’m also leaning towards @grfo’s answer as it appears that the certificate chain is not able to be verified, causing the error you are seeing.

Would indicate that you are using an Amazon root ca.

chill · July 3, 2024, 3:04am

Hi grfo,
Does it work with AWS SDK? I have tested with curl command, it could be AmazonRootCA1.pem or AmazonRootCA3.pem

curl --tlsv1.3 --cacert AmazonRootCA1.pem --cert 0d67571ca4bacc221802c0cd58c3215c349f7dd07dd3b66edc0c24c4f4de5b72-certificate.pem.crt --key 0d67571ca4bacc221802c0cd58c3215c349f7dd07dd3b66edc0c24c4f4de5b72-private.pem.key -v -X POST -d "{ \"hello\": \"world\"}" "https://xxxxxxxxx-ats.iot.ap-east-1.amazonaws.com:8443/topics/hello/world"

Message sent correctly:

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying xxx.xxx.xxx.xxx:8443...
* Connected to xxxxxxxxxx-ats.iot.ap-east-1.amazonaws.com (xxx.xxx.xxx.xxx) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: AmazonRootCA1.pem
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=*.iot.ap-east-1.amazonaws.com
*  start date: Dec 14 00:00:00 2023 GMT
*  expire date: Nov 21 23:59:59 2024 GMT
*  subjectAltName: host "xxxxxxxxxx-ats.iot.ap-east-1.amazonaws.com" matched cert's "*.iot.ap-east-1.amazonaws.com"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
*  SSL certificate verify ok.
> POST /topics/hello/world HTTP/1.1
> Host: xxxxxxxxxx-ats.iot.ap-east-1.amazonaws.com:8443
> User-Agent: curl/7.74.0
> Accept: */*
> Content-Length: 19
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 19 out of 19 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-type: application/json
< content-length: 65
< date: Wed, 03 Jul 2024 02:57:48 GMT
< x-amzn-RequestId: 4f3e9946-e3c7-d57a-89c9-beae3d0598a2
< connection: keep-alive
<
* Connection #0 to host xxxxxxxxxx-ats.iot.ap-east-1.amazonaws.com left intact
{"message":"OK","traceId":"4f3e9946-e3c7-d57a-89c9-beae3d0598a2"}

But when I test with zymkey engine

curl --tlsv1.3 --cacert AmazonRootCA1.pem --cert zymkey.crt --key nozymkey.key --engine zymkey_ssl --key-type ENG -v -X POST -d "{ \"hello\": \"world\"}" "https://xxxxxxxxx-ats.iot.ap-east-1.amazonaws.com:8443/topics/hello/world"

I got missing private key error:

* Connected to [xxxxxxxxx]-ats.iot.ap-east-1.amazonaws.com (xx.xxx.xxx.xxx) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: AmazonRootCA1.pem
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, internal error (592):
* error:100F907D:elliptic curve routines:ossl_ecdsa_sign_sig:missing private key
* Closing connection 0
curl: (35) error:100F907D:elliptic curve routines:ossl_ecdsa_sign_sig:missing private key

chill · July 3, 2024, 3:16am

Hi davidgs,
I’m not sure it’s related to the Amazon root ca. With same configuration (AmazonRootCA1.pem), it works when I use the key and cert generated by AWS.
Do you have an example implemented with pkcs11 and aws iot sdk or aws iot device client?

grfo · July 3, 2024, 5:29am

Hi chill,
With the AmazonRootCA1.pem I don’t get the TLS error, however I still don’t get a successful connect. The error now is “sign resizing because digest and key lengths (32/64) do not match”. I have no idea what is wrong.

davidgs_zymbit1 · July 3, 2024, 12:54pm

Sorry to have to drop this in the middle, but I am going to be away from the office/internet for the next 2 weeks. I’ll get someone else to step in as needed.

dg

chill · July 4, 2024, 2:15am

Hi All,
Thank you for your help. Problem resolved.
Here what I have done to make it working:

follow the tutorial AWS IoT - TLS Client Certificate Authentication | to sign a certificate
With sudo, import the zymkey into pkcs#11 PKCS#11 Support
Delete the code block from line 430 ~ 440, aws-iot-device-client/source/main.cpp at main · awslabs/aws-iot-device-client · GitHub

#else
    if (config.config.secureElement.enabled)
    {
        LOGM_ERROR(
            TAG,
            "*** %s: Secure Element configuration is enabled but feature is not compiled into binary.",
            DC_FATAL_ERROR);
        deviceClientAbort(
            "Invalid configuration. Secure Element configuration is enabled but feature is not compiled into binary.",
            EXIT_FAILURE);
    }

Compile the binary file of aws-iot-device-client by command below with debian bulleye:

# Building
git clone https://github.com/awslabs/aws-iot-device-client
cd aws-iot-device-client
mkdir build
cd build
cmake -DEXCLUDE_SECURE_ELEMENT="OFF" ../
cmake --build . --target aws-iot-device-client

Add configuration

{
    ...
    "secure-element": {
		"enabled": false,
		"pkcs11-lib": "<replace_with_pkcs11_lib_path>",
		"secure-element-pin": "<replace_with_secure_element_pin>",
		"secure-element-key-label": "<replace_with_secure_element_key_label>",
		"secure-element-slot-id": replace_with_secure_element_slot_id_integer,
		"secure-element-token-label": "<replace_with_secure_element_token_label>"
	}
    ...
}

Make sure the endpoint of IoT Core allows TLS 1.2
Execute the aws-iot-device-client with root permission
Finally, the console show the log if everything works:

sign resizing because digest and key lengths (32/64) do not match
2024-07-04T01:57:47.313Z [INFO]  {SharedCrtResourceManager.cpp}: MQTT connection established with return code: 0
2024-07-04T01:57:47.313Z [INFO]  {SharedCrtResourceManager.cpp}: Shared MQTT connection is ready!
2024-07-04T01:57:47.314Z [INFO]  {Main.cpp}: Config shadow is disabled
2024-07-04T01:57:47.314Z [INFO]  {Main.cpp}: Jobs is disabled
2024-07-04T01:57:47.314Z [INFO]  {Main.cpp}: Secure Tunneling is enabled
2024-07-04T01:57:47.314Z [INFO]  {Main.cpp}: Device Defender is disabled
2024-07-04T01:57:47.314Z [INFO]  {Main.cpp}: Sample shadow is disabled
2024-07-04T01:57:47.314Z [INFO]  {Main.cpp}: Pub Sub is disabled
2024-07-04T01:57:47.314Z [INFO]  {Main.cpp}: Sensor Publish is disabled
2024-07-04T01:57:47.314Z [INFO]  {SharedCrtResourceManager.cpp}: Starting Device Client features.
2024-07-04T01:57:47.314Z [INFO]  {SecureTunnelingFeature.cpp}: Running Secure Tunneling!
2024-07-04T01:57:47.314Z [INFO]  {Main.cpp}: Client base has been notified that Secure Tunneling has started

davidgs_zymbit1 · July 22, 2024, 2:01pm

Glad to see you were able to resolve this!

And thank you so much for coming back to post the solution! It will help others in the future I am sure!

Best Regards,
dg

Topic		Replies	Views
Options for connecting to AWS IoT ZYMKEY4 aws-iot	0	767	December 18, 2018
Zymbit Error Client Authentication ZYMKEY4 aws-iot	2	222	September 11, 2023
How to integrate Zymbit with AWS Credentials Provider GENERAL aws-iot	1	945	April 6, 2021
OpenVPN client using ZYMKEY4 ZYMKEY4	1	17	September 27, 2024
PKCS#11 Support ZYMKEY4	19	1925	July 2, 2024

Aws-iot-device-client Integration

Related topics