Skip to main content
Version: Next

Hardware Security Module (HSM)

thin-edge.io supports HSM using PKCS #11 (aka cryptoki) cryptographic tokens for MQTT client authentication between the device and the cloud.

With this feature, thin-edge.io uses an Hardware Security Module (HSM) to store the private key of the device, preventing this key to be stolen. Device authentication is then delegated by thin-edge.io to the module using the PKCS#11 protocol when a TLS connection is established.

When running tedge connect or tedge reconnect command, as part of a TLS handshake with the remote MQTT broker, a proof of ownership of the device certificate is required. This is achieved by signing a TLS 1.3 CertificateVerify message by the PKCS #11 cryptographic token. This happens only once when establishing an MQTT connection over TLS and will only need to be repeated when a new connection is opened.

Any HSM which has a PKCS#11 interface are supported, some examples of such modules are:

  • USB based devices like NitroKey HSM 2, Yubikey 5
  • TPM 2.0 (Trusted Platform Module)
  • ARM TrustZone (via OP-TEE)

For now, HSM is only used for the TLS MQTT connection between the device and C8y cloud. Additionally, the built-in bridge has to be used and the user has to device certificate corresponds to private key stored in the HSM (a step that depends on the actual key).

Configuration​

This feature has the following related configuration options:

The options below are used to configure the P11 provider, a component which directly interacts with PKCS 11 devices and makes them available to tedge and other subcomponents, as well as to select the provider that tedge will use.

There are 2 providers:

  • module: tedge will load the a P11 library directly. Can be used when PKCS 11 cryptographic tokens are directly reachable
  • socket: tedge will connect to tedge-p11-server proxy via a UNIX socket. This can be used when cryptographic tokens are not accessible from tedge, for example when running inside an isolated container.

Settings like cryptoki.uri and cryptoki.pin are the default values that the provider will use if a consumer does not provide them. In practice, this is only relevant to tedge-p11-server provider, which runs in a separate process and can handle many clients. If desired, it can be configured to use a default pin or limit the scope of tokens available to clients.

tedge config list --doc device.cryptoki
       device.cryptoki.mode  Whether to use a Hardware Security Module for authenticating the MQTT connection with the cloud.  "off" to not use the HSM, "module" to use the provided cryptoki dynamic module, "socket" to access the HSM via tedge-p11-server signing service. 
                             Examples: off, module, socket
device.cryptoki.module_path  A path to the PKCS#11 module used for interaction with the HSM.  Needs to be set when `device.cryptoki.mode` is set to `module`. 
                             Example: /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so
        device.cryptoki.pin  A default User PIN value for logging into the PKCS11 token.  May be overridden on a per-key basis using device.key_pin config setting. 
                             Example: 123456
        device.cryptoki.uri  A URI of the token/object to be used by tedge-p11-server.  If set, tedge-p11-server will by default use this URI to select a key for signing if a client does not provide its URI in the request. If the client provides the URI, then the attributes of this server URI will be used as a base onto which client-provided URI attributes will be appended, potentially limiting the scope of keys or tokens that can be used by the clients.  For example, if `cryptoki.uri=pkcs11:token=token1` and `device.key_uri=pkcs11:token2;object=key1`, `tedge-p11-server` will use URI `pkcs11:token1;object=key1`.  For more information about PKCS11 URIs, see RFC7512. 
                             Example: pkcs11:token=my-pkcs11-token;object=my-key
device.cryptoki.socket_path  A path to the tedge-p11-server socket.  Needs to be set when `device.cryptoki.mode` is set to `socket`. 
                             Example: /run/tedge-p11-server/tedge-p11-server.sock

The options below are used by the consumer (tedge) to ask tedge-p11-server for a specific key. You can use different keys for different connection profiles. tedge-p11-server may limit what keys and tokens are available.

tedge config list --doc key_uri
    device.key_uri  A PKCS#11 URI of the private key.  See RFC #7512.
                    Example: pkcs11:token=my-pkcs11-token;object=my-key

c8y.device.key_uri  A PKCS#11 URI of the private key.  See RFC #7512.
                    Example: pkcs11:token=my-pkcs11-token;object=my-key

 az.device.key_uri  A PKCS#11 URI of the private key.  See RFC #7512.
                    Example: pkcs11:token=my-pkcs11-token;object=my-key

aws.device.key_uri  A PKCS#11 URI of the private key.  See RFC #7512.
                    Example: pkcs11:model=PKCS%2315%20emulated

The options below are used by the consumer (tedge) to use a given PIN with a given key, instead of using a default PIN that tedge-p11-server is configured to use.

tedge config list --doc key_pin
    device.key_pin  User PIN value for logging into the PKCS#11 token provided by the consumer.  This differs from cryptoki.pin in that cryptoki.pin is used by PKCS#11 provider, e.g. tedge-p11-server as a default PIN for all tokens, but device.key_pin is the PIN provided by the consumer (tedge) with a given `key_uri`.  In practice, this can be used to define separate keys and separate PINs for different connection profiles.
                    Examples: 123456, my-pin
c8y.device.key_pin  User PIN value for logging into the PKCS#11 token provided by the consumer.  This differs from cryptoki.pin in that cryptoki.pin is used by PKCS#11 provider, e.g. tedge-p11-server as a default PIN for all tokens, but device.key_pin is the PIN provided by the consumer (tedge) with a given `key_uri`.  In practice, this can be used to define separate keys and separate PINs for different connection profiles.
                    Examples: 123456, my-pin
 az.device.key_pin  User PIN value for logging into the PKCS#11 token provided by the consumer.  This differs from cryptoki.pin in that cryptoki.pin is used by PKCS#11 provider, e.g. tedge-p11-server as a default PIN for all tokens, but device.key_pin is the PIN provided by the consumer (tedge) with a given `key_uri`.  In practice, this can be used to define separate keys and separate PINs for different connection profiles.
                    Examples: 123456, my-pin
aws.device.key_pin  User PIN value for logging into the PKCS#11 token provided by the consumer.  This differs from cryptoki.pin in that cryptoki.pin is used by PKCS#11 provider, e.g. tedge-p11-server as a default PIN for all tokens, but device.key_pin is the PIN provided by the consumer (tedge) with a given `key_uri`.  In practice, this can be used to define separate keys and separate PINs for different connection profiles.
                    Examples: 123456, my-pin

Setup guide​

The following guide shows how to connect to Cumulocity using a PKCS #11 cryptographic token. Instead of using a dedicated hardware token, we'll create a software token using SoftHSM2 and import currently used private key on it.

note

While this guide uses SoftHSM2 to demonstrate the feature, be aware that in a real production setting you'll probably be using a different, hardware token. The process of setting up the token itself may be different for each token type, as well as require using a different PKCS #11 dynamic library, but in all cases, the goal is to:

  • store the private key on the HSM
  • store the corresponding certificate on the file system
  • set device.cert_path to the certificate path
  • set device.cryptoki.module_path to the correct PKCS #11 dynamic library
  • set device.cryptoki.pin and device.cryptoki.uri accordingly to the local HSM settings

Step 1: Setup the cryptographic token​

  1. Install SoftHSM2 (to create the token and key) and p11tool (to view the PKCS #11 URI of the key).

    sudo apt-get install -y softhsm2 gnutls-bin

For SoftHSM configuration, see SoftHSM README.

  1. Add tedge and current user to softhsm group. Only users belonging to softhsm group can view and manage SoftHSM tokens. After adding your own user, remember to logout and login for changes to take effect. Alternatively, you can just run softhsm2-util and p11tool with sudo.

     sudo usermod -a -G softhsm tedge
     sudo usermod -a -G softhsm $(id -un)

  2. Create a new SoftHSM token. You'll be prompted for a PIN for a regular user and security officer (SO). The rest of the guide assumes PIN=123456, but you're free to use a different one.

    softhsm2-util --init-token --slot 0 --label my-token

  3. Import the private key to the created token. Make sure to use the correct PIN value for a regular user from the previous step.

    PUB_PRIV_KEY=$(
        cat "$(tedge config get device.key_path)" && cat "$(tedge config get device.cert_path)"
    )
    softhsm2-util \
        --import <(echo "$PUB_PRIV_KEY") \
        --token my-token \
        --label my-key \
        --id 01 \
        --pin 123456 \

  4. Get the URI of the key

    First, see what tokens are available

    p11tool --list-tokens
    Output
    ...
    Token 2:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=83f9cf49039c051a;token=my-token
        Label: my-token
        Type: Generic token
        Flags: RNG, Requires login
        Manufacturer: SoftHSM project
        Model: SoftHSM v2
        Serial: 83f9cf49039c051a
        Module: /usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so
    ...

    Now check if the private key object is in the token. You may need to login, provide the regular user PIN and also provide token URL(URI) if multiple tokens are connected:

    p11tool --login --set-pin=123456 --list-privkeys "pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=83f9cf49039c051a;token=my-token"
    Output
    Object 0:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=83f9cf49039c051a;token=my-token;id=%01;object=my-key;type=private
        Type: Private key (EC/ECDSA-SECP256R1)
        Label: my-key
        Flags: CKA_PRIVATE; CKA_SENSITIVE;
        ID: 01

Step 2: thin-edge.io setup​

Next, we're going to configure tedge to use the token directly using module mode.

If that mode doesn't work for you, because of the token can't be accessed for some reason or you can't dynamically load the PKCS #11 library, see how to use tedge-p11-server.

Using the module mode, the cryptoki module will be loaded by the dynamic loader and used for signing. If there are many tokens or private keys we also need to provide the URI for the key to select a correct one.

  1. Enable the module mode and set the module and the key URI.

    tedge config set device.cryptoki.mode module
    tedge config set device.cryptoki.module_path /usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so
    tedge config set device.key_uri "pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=83f9cf49039c051a;token=my-token;id=%01;object=my-key;type=private"
    note

    [cloud].device.key_uri config setting corresponds to the usual [cloud].device.key_path setting, but instead of pointing to the private key file, it contains the URI for a given cloud.

Step 3: Reconnect​

  1. Enable the built-in bridge. PKCS #11 doesn't work when using mosquitto as a bridge.

    tedge config set mqtt.bridge.built_in true

  2. Reconnect to c8y

    tedge reconnect c8y
    tedge reconnect c8y
    Disconnecting from Cumulocity
    Removing bridge config file... ✓
    Disabling tedge-mapper-c8y... ✓
    reconnect to Cumulocity cloud.:
            device id: marcel-hsm-device-rsa
            cloud profile: <none>
            cloud host: thin-edge-io.eu-latest.cumulocity.com:8883
            auth type: Certificate
            certificate file: /etc/tedge/device-certs/rsa/tedge-certificate.pem
            cryptoki: true
            bridge: built-in
            service manager: systemd
            mosquitto version: 2.0.20
            proxy: Not configured
    Creating device in Cumulocity cloud... ✓
    Restarting mosquitto... ✓
    Waiting for mosquitto to be listening for connections... ✓
    Enabling tedge-mapper-c8y... ✓
    Verifying device is connected to cloud... ✓
    Checking Cumulocity is connected to intended tenant... ✓
    Enabling tedge-agent... ✓

    cryptoki: true in the connection summary confirms that we connected using our PKCS #11 token.

Key selection​

tedge or tedge-p11-server will try to find a private key even if the URI is not provided. In cases where there are multiple tokens/keys to choose from, the first one returned by the system will be automatically selected, but appropriate warning will be emitted:

WARN tedge_p11_server::pkcs11: Multiple keys were found. If the wrong one was chosen, please use a URI that uniquely identifies a key.

In such cases, config setting device.key_uri can be used to select an appropriate key or token on which the key is located.

It is also possible to use a URI that identifies a token in settings like device.key_uri. The URI will then be used to select a token, but the key will be selected automatically, though the selected key may be wrong if there are multiple to choose from. Also if the URI contain attributes that identify a key, but doesn't contain attributes that identify a token, still the first token will be selected, even if another token contains the intended key.