Skip to main content
Version: Next

Hardware Security Module (HSM)

thin-edge.io supports HSM using PKCS #11 (aka cryptoki) cryptographic tokens for MQTT client authentication between the device and the cloud.

With this feature, thin-edge.io uses an Hardware Security Module (HSM) to store the private key of the device, preventing this key to be stolen. Device authentication is then delegated by thin-edge.io to the module using the PKCS#11 protocol when a TLS connection is established.

When running tedge connect or tedge reconnect command, as part of a TLS handshake with the remote MQTT broker, a proof of ownership of the device certificate is required. This is achieved by signing a TLS 1.3 CertificateVerify message by the PKCS #11 cryptographic token. This happens only once when establishing an MQTT connection over TLS and will only need to be repeated when a new connection is opened.

Any HSM which has a PKCS#11 interface are supported, some examples of such modules are:

  • USB based devices like NitroKey HSM 2, Yubikey 5
  • TPM 2.0 (Trusted Platform Module)
  • ARM TrustZone (via OP-TEE)

For now, HSM is only used for the TLS MQTT connection between the device and C8y cloud. Additionally, the built-in bridge has to be used and the user has to device certificate corresponds to private key stored in the HSM (a step that depends on the actual key).

Configuration​

This feature has the following related configuration options:

The options below are used to configure the P11 provider, a component which directly interacts with PKCS 11 devices and makes them available to tedge and other subcomponents, as well as to select the provider that tedge will use.

There are 2 providers:

  • module: tedge will load the a P11 library directly. Can be used when PKCS 11 cryptographic tokens are directly reachable
  • socket: tedge will connect to tedge-p11-server proxy via a UNIX socket. This can be used when cryptographic tokens are not accessible from tedge, for example when running inside an isolated container.

Settings like cryptoki.uri and cryptoki.pin are the default values that the provider will use if a consumer does not provide them. In practice, this is only relevant to tedge-p11-server provider, which runs in a separate process and can handle many clients. If desired, it can be configured to use a default pin or limit the scope of tokens available to clients.

tedge config list --doc device.cryptoki
       device.cryptoki.mode  Whether to use a Hardware Security Module for authenticating the MQTT connection with the cloud.  "off" to not use the HSM, "module" to use the provided cryptoki dynamic module, "socket" to access the HSM via tedge-p11-server signing service. 
                             Examples: off, module, socket
device.cryptoki.module_path  A path to the PKCS#11 module used for interaction with the HSM.  Needs to be set when `device.cryptoki.mode` is set to `module`. 
                             Example: /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so
        device.cryptoki.pin  A default User PIN value for logging into the PKCS11 token.  May be overridden on a per-key basis using device.key_pin config setting. 
                             Example: 123456
        device.cryptoki.uri  A URI of the token/object to be used by tedge-p11-server.  If set, tedge-p11-server will by default use this URI to select a key for signing if a client does not provide its URI in the request. If the client provides the URI, then the attributes of this server URI will be used as a base onto which client-provided URI attributes will be appended, potentially limiting the scope of keys or tokens that can be used by the clients.  For example, if `cryptoki.uri=pkcs11:token=token1` and `device.key_uri=pkcs11:token2;object=key1`, `tedge-p11-server` will use URI `pkcs11:token1;object=key1`.  For more information about PKCS11 URIs, see RFC7512. 
                             Example: pkcs11:token=my-pkcs11-token;object=my-key
device.cryptoki.socket_path  A path to the tedge-p11-server socket.  Needs to be set when `device.cryptoki.mode` is set to `socket`. 
                             Example: /run/tedge-p11-server/tedge-p11-server.sock

The options below are used by the consumer (tedge) to ask tedge-p11-server for a specific key. You can use different keys for different connection profiles. tedge-p11-server may limit what keys and tokens are available.

tedge config list --doc key_uri
    device.key_uri  A PKCS#11 URI of the private key.  See RFC #7512.
                    Example: pkcs11:token=my-pkcs11-token;object=my-key

c8y.device.key_uri  A PKCS#11 URI of the private key.  See RFC #7512.
                    Example: pkcs11:token=my-pkcs11-token;object=my-key

 az.device.key_uri  A PKCS#11 URI of the private key.  See RFC #7512.
                    Example: pkcs11:token=my-pkcs11-token;object=my-key

aws.device.key_uri  A PKCS#11 URI of the private key.  See RFC #7512.
                    Example: pkcs11:model=PKCS%2315%20emulated

The options below are used by the consumer (tedge) to use a given PIN with a given key, instead of using a default PIN that tedge-p11-server is configured to use.

tedge config list --doc key_pin
    device.key_pin  User PIN value for logging into the PKCS#11 token provided by the consumer.  This differs from cryptoki.pin in that cryptoki.pin is used by PKCS#11 provider, e.g. tedge-p11-server as a default PIN for all tokens, but device.key_pin is the PIN provided by the consumer (tedge) with a given `key_uri`.  In practice, this can be used to define separate keys and separate PINs for different connection profiles. 
                    Examples: 123456, my-pin

c8y.device.key_pin  User PIN value for logging into the PKCS#11 token provided by the consumer.  This differs from cryptoki.pin in that cryptoki.pin is used by PKCS#11 provider, e.g. tedge-p11-server as a default PIN for all tokens, but device.key_pin is the PIN provided by the consumer (tedge) with a given `key_uri`.  In practice, this can be used to define separate keys and separate PINs for different connection profiles. 
                    Examples: 123456, my-pin

 az.device.key_pin  User PIN value for logging into the PKCS#11 token provided by the consumer.  This differs from cryptoki.pin in that cryptoki.pin is used by PKCS#11 provider, e.g. tedge-p11-server as a default PIN for all tokens, but device.key_pin is the PIN provided by the consumer (tedge) with a given `key_uri`.  In practice, this can be used to define separate keys and separate PINs for different connection profiles. 
                    Examples: 123456, my-pin

aws.device.key_pin  User PIN value for logging into the PKCS#11 token provided by the consumer.  This differs from cryptoki.pin in that cryptoki.pin is used by PKCS#11 provider, e.g. tedge-p11-server as a default PIN for all tokens, but device.key_pin is the PIN provided by the consumer (tedge) with a given `key_uri`.  In practice, this can be used to define separate keys and separate PINs for different connection profiles. 
                    Examples: 123456, my-pin
note

Not every token is protected by a PIN. If a token reports that no login is required (the CKF_LOGIN_REQUIRED flag is not set), thin-edge.io uses it without logging in, and both device.cryptoki.pin and device.key_pin are ignored for that token. Such a token has no user PIN to manage either, so tedge hsm change-pin reports that there is nothing to change, and tedge hsm init treats an already initialized token as ready to use.

Setup guide​

The following guide shows how to connect to Cumulocity using a PKCS #11 cryptographic token. Instead of using a dedicated hardware token, we'll create a software token using SoftHSM2 and import currently used private key on it.

note

While this guide uses SoftHSM2 to demonstrate the feature, be aware that in a real production setting you'll probably be using a different, hardware token. The process of setting up the token itself may be different for each token type, as well as require using a different PKCS #11 dynamic library, but in all cases, the goal is to:

  • store the private key on the HSM
  • store the corresponding certificate on the file system
  • set device.cert_path to the certificate path
  • set device.cryptoki.module_path to the correct PKCS #11 dynamic library
  • set device.cryptoki.pin and device.cryptoki.uri accordingly to the local HSM settings

Step 1: Setup the cryptographic token​

  1. Install SoftHSM2 (to create the token and key) and p11tool (to view the PKCS #11 URI of the key).

    sudo apt-get install -y softhsm2 gnutls-bin

For SoftHSM configuration, see SoftHSM README.

  1. Add tedge and current user to softhsm group. Only users belonging to softhsm group can view and manage SoftHSM tokens. After adding your own user, remember to logout and login for changes to take effect. Alternatively, you can just run softhsm2-util and p11tool with sudo.

     sudo usermod -a -G softhsm tedge
     sudo usermod -a -G softhsm $(id -un)

  2. Create a new SoftHSM token. You'll be prompted for a PIN for a regular user and security officer (SO). The rest of the guide assumes PIN=123456, but you're free to use a different one.

    softhsm2-util --init-token --slot 0 --label my-token

  3. Import the private key to the created token. Make sure to use the correct PIN value for a regular user from the previous step.

    PUB_PRIV_KEY=$(
        cat "$(tedge config get device.key_path)" && cat "$(tedge config get device.cert_path)"
    )
    softhsm2-util \
        --import <(echo "$PUB_PRIV_KEY") \
        --token my-token \
        --label my-key \
        --id 01 \
        --pin 123456 \

  4. Get the URI of the key

    First, see what tokens are available

    p11tool --list-tokens
    Output
    ...
    Token 2:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=83f9cf49039c051a;token=my-token
        Label: my-token
        Type: Generic token
        Flags: RNG, Requires login
        Manufacturer: SoftHSM project
        Model: SoftHSM v2
        Serial: 83f9cf49039c051a
        Module: /usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so
    ...

    Now check if the private key object is in the token. You may need to login, provide the regular user PIN and also provide token URL(URI) if multiple tokens are connected:

    p11tool --login --set-pin=123456 --list-privkeys "pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=83f9cf49039c051a;token=my-token"
    Output
    Object 0:
        URL: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=83f9cf49039c051a;token=my-token;id=%01;object=my-key;type=private
        Type: Private key (EC/ECDSA-SECP256R1)
        Label: my-key
        Flags: CKA_PRIVATE; CKA_SENSITIVE;
        ID: 01

Step 2: thin-edge.io setup​

Next, we're going to configure tedge to use the token directly using module mode.

If that mode doesn't work for you, because of the token can't be accessed for some reason or you can't dynamically load the PKCS #11 library, see how to use tedge-p11-server.

Using the module mode, the cryptoki module will be loaded by the dynamic loader and used for signing. If there are many tokens or private keys we also need to provide the URI for the key to select a correct one.

  1. Enable the module mode and set the module and the key URI.

    tedge config set device.cryptoki.mode module
    tedge config set device.cryptoki.module_path /usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so
    tedge config set device.key_uri "pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=83f9cf49039c051a;token=my-token;id=%01;object=my-key;type=private"
    note

    [cloud].device.key_uri config setting corresponds to the usual [cloud].device.key_path setting, but instead of pointing to the private key file, it contains the URI for a given cloud.

Step 3: Reconnect​

  1. Enable the built-in bridge. PKCS #11 doesn't work when using mosquitto as a bridge.

    tedge config set mqtt.bridge.built_in true

  2. Reconnect to c8y

    tedge reconnect c8y
    tedge reconnect c8y
    Disconnecting from Cumulocity
    Removing bridge config file... ✓
    Disabling tedge-mapper-c8y... ✓
    reconnect to Cumulocity cloud.:
            device id: marcel-hsm-device-rsa
            cloud profile: <none>
            cloud host: thin-edge-io.eu-latest.cumulocity.com:8883
            auth type: Certificate
            certificate file: /etc/tedge/device-certs/rsa/tedge-certificate.pem
            cryptoki: true
            bridge: built-in
            service manager: systemd
            mosquitto version: 2.0.20
            proxy: Not configured
    Creating device in Cumulocity cloud... ✓
    Restarting mosquitto... ✓
    Waiting for mosquitto to be listening for connections... ✓
    Enabling tedge-mapper-c8y... ✓
    Verifying device is connected to cloud... ✓
    Checking Cumulocity is connected to intended tenant... ✓
    Enabling tedge-agent... ✓

    cryptoki: true in the connection summary confirms that we connected using our PKCS #11 token.

Key selection​

tedge or tedge-p11-server will try to find a private key even if the URI is not provided. In cases where there are multiple tokens/keys to choose from, the first one returned by the system will be automatically selected, but appropriate warning will be emitted:

WARN tedge_p11_server::pkcs11: Multiple keys were found. If the wrong one was chosen, please use a URI that uniquely identifies a key.

In such cases, config setting device.key_uri can be used to select an appropriate key or token on which the key is located.

It is also possible to use a URI that identifies a token in settings like device.key_uri. The URI will then be used to select a token, but the key will be selected automatically, though the selected key may be wrong if there are multiple to choose from. Also if the URI contain attributes that identify a key, but doesn't contain attributes that identify a token, still the first token will be selected, even if another token contains the intended key.

Token initialization​

Typically a token must be initialized before it can hold any keys. The tedge hsm init command runs the PKCS #11 initialization sequence, so the only thing you have to know about the device is the module path (device.cryptoki.module_path) that points to the PKCS #11 dynamic library.

In most cases the command can be run without any arguments:

tedge hsm init

The command prints the resulting token URI to stdout (and human-readable status to stderr):

Output
pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=a30ed1ca6244fc5f;token=tedge
note

Other tools, such as p11tool, can also be used to initialize a token if required, for example when the token needs vendor specific initialization options.

Without arguments, the following defaults are used:

  • the slot is selected automatically: an uninitialized token is initialized, or, if there is none, an already usable token is reused. If there are several candidates to choose from, the command fails rather than guessing, and the slot has to be selected explicitly
  • the token is labelled tedge
  • the user PIN, used by all subsequent operations, is the PIN configured for tedge-p11-server (device.cryptoki.pin)
  • the Security Officer (SO) PIN, only needed to initialize the token, is the same as the user PIN, which works for tokens that do not enforce distinct PINs (such as SoftHSM2)

Each of these can be overridden if the token requires it. The slot to initialize is selected by passing a PKCS #11 URI, as printed by tedge hsm list-tokens. A token only gets a label and a serial once it is initialized, so an uninitialized slot is addressed by its model and slot id:

tedge hsm init --label my-token --pin 123456 --so-pin 654321 "pkcs11:model=SoftHSM%20v2;slot-id=1"
note

Slot ids are assigned by the PKCS #11 module and can change, for instance when another token is initialized. Read the URI from tedge hsm list-tokens right before using it, rather than storing it.

The command is idempotent: if a token with the requested label is already initialized, it is left untouched and its URI is returned, and duplicate labels are never created. A given URI must match a token though, otherwise the command fails. In particular, since slot ids can change once a token is initialized, re-running the same command with a stored slot URI fails once the URI has gone stale; re-runnable scripts should omit the URI and rely on the label, or read the URI from tedge hsm list-tokens each time.

Because the URI is the only thing written to stdout, the command is easy to use in scripts:

TOKEN_URI=$(tedge hsm init --label my-token)
tedge hsm create-key --type ecdsa "$TOKEN_URI"
note

tedge hsm create-key also initializes a token automatically when no initialized token exists yet (see below), so on a fresh device with a single HSM you can go straight from setting the module path to creating a key in a single command.

Key generation​

tedge hsm create-key --help
Generate a new keypair on the PKCS #11 token and select it to be used.

Can be used to generate a keypair on the TOKEN. If the TOKEN argument is not provided, the command auto-discovers the token to use: if no initialized token exists yet, an uninitialized slot is initialized automatically; if exactly one initialized token exists, it is used; if several exist, the available tokens are printed so one can be selected.

The command generates an RSA or an ECDSA keypair on the token. When using RSA, `--bits` is used to set the size of the key, when using ECDSA, `--curve` is used.

The command is idempotent: if a key matching the given label (and id, if provided) already exists on the token, it is reused instead of creating a duplicate. Pass `--force-new` to always generate a new key.

After the key is generated (or reused), tedge config is updated to use the key using the `device.key_uri` property. Depending on the selected cloud, we use `device.key_uri` setting for that cloud, e.g. `create-key c8y` will write to `c8y.device.key_uri`. However, if the setting already points to a different key that exists on the token, it is left unchanged so a working key selection is never silently replaced; the `tedge config set` command to switch to the new key is printed instead.

Usage: tedge hsm create-key [OPTIONS] [TOKEN] [COMMAND]

Commands:
  c8y   
  az    
  aws   
  help  Print this message or the help of the given subcommand(s)

Arguments:
  [TOKEN]
          The URI of the token where the keypair should be created.
          
          If this argument is missing, the token is auto-discovered: an uninitialized token is initialized automatically, a single initialized token is used as-is, and if several exist the available tokens are listed so one can be selected.

Options:
      --config-dir <CONFIG_DIR>
          [env: TEDGE_CONFIG_DIR, default: /etc/tedge]

      --label <LABEL>
          Human readable description (CKA_LABEL attribute) for the key
          
          [default: tedge]

      --debug
          Turn-on the DEBUG log level.
          
          If off only reports ERROR, WARN, and INFO, if on also reports DEBUG

      --id <ID>
          Key identifier for the keypair (CKA_ID attribute).
          
          If provided and no object exists on the token with the same ID, this will be the ID of the new keypair. If an object with this ID already exists, the operation will return an error. If not provided, a random ID will be generated and used by the keypair.
          
          The id shall be provided as hex digits without `0x` prefix, e.g. `--id 010203`. Byte separators as printed by other tools (e.g. `01:02:03`) are accepted as well.

      --log-level <LOG_LEVEL>
          Configures the logging level.
          
          One of error/warn/info/debug/trace. Logs with verbosity lower or equal to the selected level will be printed, i.e. warn prints ERROR and WARN logs and trace prints logs of all levels.
          
          Overrides `--debug`

      --type <TYPE>
          The type of the key
          
          [default: ecdsa]
          [possible values: rsa, ecdsa]

      --bits <BITS>
          The size of the RSA keys in bits. Should only be used with --type rsa
          
          [default: 2048]
          [possible values: 2048, 3072, 4096]

      --curve <CURVE>
          The curve (size) of the ECDSA key. Should only be used with --type ecdsa
          
          [default: p256]
          [possible values: p256, p384]

      --pin <PIN>
          User PIN value for logging into the PKCS #11 token.
          
          This flag can be used to provide a PIN when creating a new key without needing to update tedge-config, which can be helpful when initializing keys on new tokens.
          
          Note that in contrast to the URI of the key, which will be written to tedge-config automatically when the keypair is created, PIN will not be written automatically and may be needed to written manually using tedge config set (if not using tedge-p11-server with the correct default PIN).

      --so-pin <SO_PIN>
          Security Officer (SO) PIN used if a token has to be initialized automatically.
          
          When no initialized token exists, an uninitialized slot is initialized automatically before the key is created. The SO PIN is required to initialize a token; if not provided, the user PIN is used as the SO PIN (which works for tokens that do not enforce distinct PINs, such as SoftHSM2).

      --token-label <TOKEN_LABEL>
          Label (CKA_LABEL) to assign to the token if one has to be initialized automatically
          
          [default: tedge]

      --force-new
          Always create a new key, even if one with the same label already exists.
          
          By default the command is idempotent: if a key matching the given label (and id, if provided) already exists on the token, it is reused instead of creating a duplicate. Pass this flag to force generating a new key regardless.

      --outfile-pubkey <OUTFILE_PUBKEY>
          Path where public key will be saved when a keypair is generated

  -h, --help
          Print help (see a summary with '-h')

tedge hsm create-key command generates a new keypair on the PKCS #11 token.

  1. Configure cryptoki in module or socket mode as described in previous sections.

  2. Run the tedge hsm create-key command. You'll need to provide key type, size and label of the key object.

    tedge hsm create-key --type ecdsa --curve p256 --label my-key
    Output
    New keypair was successfully created.
    Key URI: pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=a30ed1ca6244fc5f;token=test-token;id=%51%05%87%75%6F%B7%28%EC%5E%5D%1F%B8%EB%CF%FD%96%B7%E4%28%B6;object=my-key
    Public key:
    -----BEGIN PUBLIC KEY-----
    BEsjmiXDdko90IDdjlAb/bWyTf6kd6S+/KPlj2Yd3zjHZe54evLyHJ1e8dSDhpy7
    2Tcml9ZcHWBHA+MM0NFAbaw=
    -----END PUBLIC KEY-----


    Value of `device.key_uri` was updated to point to the new key

  3. Run tedge config get device.key_uri to confirm tedge will use the new key.

    tedge config get device.key_uri
    Output
    pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=a30ed1ca6244fc5f;token=test-token;id=%51%05%87%75%6F%B7%28%EC%5E%5D%1F%B8%EB%CF%FD%96%B7%E4%28%B6;object=my-key

Now you're free to use the new key to either request a signed certificate using a CSR or to create a self-signed certificate.

note

tedge cert download c8y creates a key automatically (as if running tedge hsm create-key, including initializing the token if needed) when the PKCS #11 token verifiably holds no private key yet, so a fresh device with a single HSM can be provisioned with just:

tedge cert download c8y --device-id "$DEVICE_ID" --one-time-password "$PASSWORD"

The automatic key creation stays predictable by only handling the unambiguous case: if the HSM can't be inspected (e.g. due to a wrong PIN), if the token holds keys but none matches the configured key_uri, or if several tokens are available, the command fails and lists the options instead of guessing.