Store HSM on Thales CipherTrust Manager

MinIO KMS supports using a symmetric key in Thales CipherTrust Manager, formerly Gemalto KeySecure, as the Hardware Security Module (HSM) that protects the Root Encryption Key (REK). MinIO KMS calls this backend hsm:thales:ciphertrust.

K e y D a t a b a s e R D e e a c d r y e p n t c r D y B p t w e i d t h R o R o o t o t K e K y e y M i n I O K M S D e c r y p t u P s l i a n i g n t t e h x e t C R i o p o h t e r K T e r y u s t A E S k e y C i p h e r T r u s t M a n a g e r

CipherTrust Manager holds the key material, so a user with access to a MinIO KMS host has no immediate access to the plaintext key value. You can enable an external HSM for MinIO KMS at any time after completing the initial installation.

The CipherTrust key performs two jobs for the cluster:

  • It seals and unseals the Root Encryption Key. On startup, each node asks CipherTrust Manager to unwrap the REK, and only then can it decrypt its enclaves and the keys they hold.

  • It derives the internode TLS identity. Each node derives its private key from the same CipherTrust key, so nodes that share the key derive matching identities and trust each other.

Because the second job depends on the key material, every node in the cluster must use the same endpoint, domain, and key name.

What MinIO KMS sends to CipherTrust Manager

MinIO KMS never sends the data it protects to CipherTrust Manager. It sends only short, fixed-size values, and it uses three REST endpoints:

Endpoint Used for Details
/api/v1/auth/tokens/ Login Exchanges the configured username and password for a short-lived JSON Web Token (JWT). MinIO KMS logs in per operation and does not cache the token on disk.
/api/v1/crypto/encrypt Seal and key derivation AES-GCM wraps a fresh 256-bit data key for each seal operation. AES-ECB encrypts one fixed block to derive the node identity.
/api/v1/crypto/decrypt Unseal AES-GCM unwraps the data key, which MinIO KMS then uses to decrypt the sealed REK locally.

Sealing uses envelope encryption. MinIO KMS generates a random AES-256 data key, asks CipherTrust Manager to wrap it, then encrypts the payload locally with that data key. The response records the key version used, so a rotated key still unwraps older ciphertexts.

Prerequisites

This procedure assumes the following installations:

  • The local or cluster MinIO KMS deployment
  • A CipherTrust Manager deployment reachable from every MinIO KMS node

You also need permission to create a user and a key in the CipherTrust Manager domain you intend to use, and the CA certificate that issued the CipherTrust Manager TLS certificate unless a public CA issued it.

Refer to the Thales CipherTrust Manager administrator guide for guidance on installing CipherTrust Manager and on creating users, keys, and key policies.

See the installation instructions for further guidance on deploying MinIO KMS.

Losing the CipherTrust key causes permanent data loss
The CipherTrust key protects the entire MinIO KMS deployment. If the key is deleted or destroyed, no node can unseal the REK through CipherTrust Manager. Unless another HSM entry or a usable backup can still unseal the REK, every node and every backup becomes permanently unreadable, along with all data encrypted under the keys MinIO KMS manages. Back up the key according to your Thales operational procedures, and consider configuring multiple HSMs to mitigate the risk.

Procedure

  1. Select the CipherTrust Manager domain

    Sign in to the CipherTrust Manager console. If your deployment is split into domains, switch to the domain that holds the MinIO KMS user and key, and note the domain name for the domain field in the configuration. Leave that field empty to use the root domain.

  2. Create a user for MinIO KMS

    Create a local user, for example minkms, or reuse an existing one. Sign in as that user once, because CipherTrust Manager requires a password change on first login and MinIO KMS cannot complete that exchange. Record the final password.

    Give MinIO KMS its own user rather than sharing an operator account. You can then rotate its password, audit its requests, and revoke its access on its own.

  3. Create the encryption key

    Create a symmetric key for MinIO KMS. MinIO recommends an AES 256 key.

    The key must allow the Encrypt and Decrypt operations, and the MinIO KMS user must be permitted to use it, either as the key owner or through a key policy. Grant the user those two operations on this key only, because MinIO KMS needs no other permission in CipherTrust Manager.

    Note the key name, not its UUID. It goes into the key field of the configuration.

  4. Modify the configuration file for the MinIO KMS cluster

    Open the configuration file in your preferred text editor and add the hsm.thales.ciphertrust_manager section:

    version: v1
    
    # Other configuration settings above this line
    
    tls:
      certs:
      - key:  /etc/minkms/private.key
        cert: /etc/minkms/public.crt
      ca:   /etc/minkms/CAs   # Must trust the CipherTrust Manager CA
    
    hsm:
      thales:
        ciphertrust_manager:
          server:   https://ciphertrust.cluster.local # CipherTrust Manager endpoint
          key:      minkms-sealing-key                # Name of the symmetric key created above
          username: minkms                            # User account for accessing CipherTrust Manager
          password: "Nh[a7*J)oRbXR$wi74"              # Password for the user account
          domain:   ""                                # Optional. Empty selects the root domain
    

    MinIO KMS verifies the CipherTrust Manager TLS certificate and requires TLS 1.2 or later. If a private CA issued that certificate, add the CA to the tls.ca path, which may be a single file or a directory of PEM certificates. When tls.ca is unset, MinIO KMS also reads the CAs directory inside the node’s data directory. An untrusted certificate produces a login failure at startup.

    The configuration file holds the CipherTrust Manager password in cleartext. Restrict the file to the user the MinIO KMS server runs as, for example with chmod 600, and keep it out of version control.

    MinIO KMS sends every request over TLS unless the server value starts with http://, which sends the username and password in the clear. Use plain HTTP only in a throwaway lab.

    Make the same changes to all MinIO KMS nodes in the cluster deployment.

  5. Restart the MinIO KMS process

    Restart the nodes one at a time using systemctl restart minkms, and wait for each node to rejoin before moving to the next. Monitor the system logs using journalctl -uf minkms to ensure successful startup and resumption of internode and client API operations.

    A restart affects availability. MinIO KMS serves reads with a read quorum of 1 of n nodes, so encryption, decryption, and data key generation continue while one node is down. Writes need n of n nodes, so creating or deleting keys, identities, policies, and enclaves fails until every node is back. Restart during a maintenance window if your applications perform those write operations.

    On startup, each node prints its HSM type:

    HSM            hsm:thales:ciphertrust
    

    This line confirms only that the node loaded the CipherTrust configuration. It does not mean CipherTrust Manager holds a sealed copy of the REK, which the next step verifies.

  6. Verify the sealed key entry

    Use the minkms ls-hsm command to list the HSMs the cluster knows about and which of them hold a sealed copy of the REK.

    If CipherTrust Manager appears as configured but without an entry, for example because you added it to an existing cluster, create the entry:

    minkms add-hsm hsm:thales:ciphertrust
    

    This command requires SysAdmin privileges, and the backend must be configured on all nodes first.

    Do not continue until minkms ls-hsm shows a sealed REK entry for hsm:thales:ciphertrust. Until it does, CipherTrust Manager cannot unseal the key database, and the local HSM is the only thing keeping the cluster readable.

  7. (Optional) Disable the local HSM

    You can disable the local HSM used to initialize the cluster MinIO KMS after CipherTrust Manager holds a sealed REK entry. This prevents using that HSM or its associated REK for accessing the encryption key database.

    Complete the verification step above first. Removing the local HSM key while CipherTrust Manager holds no sealed REK entry leaves the key database permanently unreadable.

    Record the MINIO_KMS_HSM_KEY value somewhere secure before you remove it. It is the only way back if CipherTrust Manager cannot unseal the key database.

    Open the MinIO KMS environment file at /etc/default/minkms in your preferred text editor. Remove the MINIO_KMS_HSM_KEY line on one node and restart that node with systemctl restart minkms. Continue only after that node unseals the key database with CipherTrust Manager and resumes normal operation. Then repeat on the remaining nodes, one at a time.

Rotate the CipherTrust key

MinIO KMS records the key version in every sealed value, so rotating the key in CipherTrust Manager does not break existing ciphertexts. To move the cluster onto new key material:

  1. Rotate the key in CipherTrust Manager.

  2. Re-seal the REK with the new material:

    minkms add-hsm --force hsm:thales:ciphertrust
    
  3. Restart all nodes, because rotating the key also changes the internode identity each node derives from it. Restart one node at a time and wait for each to rejoin. The cluster converges once every node uses the new identity, and write operations pause until then, as described in the restart step above.

  4. Keep the old key material until nothing needs it.

    Every sealed value records the key version that wrapped it, so the old version stays necessary for as long as a sealed value still references it. Backups taken before the rotation hold a REK sealed under the old version.

    Deactivate or delete the old version only after you confirm that every sealed value you intend to keep was re-sealed under the new version, and a recovery test restores the cluster with the old version unavailable.

To rotate the password instead, change it in CipherTrust Manager, update the password field on every node, then restart the nodes. MinIO KMS uses the password only to obtain a login token, so changing it does not affect sealed data.

Remove the backend

To stop using CipherTrust Manager, first confirm with minkms ls-hsm that at least one other HSM holds a sealed REK entry. Then remove the CipherTrust entry and delete the hsm.thales.ciphertrust_manager section from every node’s configuration file:

minkms rm-hsm hsm:thales:ciphertrust

Restart every node with systemctl restart minkms, one at a time, waiting for each node to rejoin before restarting the next. MinIO KMS reads the HSM configuration only at startup, so a running node keeps its connection to CipherTrust Manager until it restarts. Write operations pause during the restarts, as described in the restart step above.

Removing the last remaining entry leaves the cluster unable to unseal its state.

Troubleshooting

The server logs the underlying CipherTrust Manager error message. Read the log of the node that fails to start before you change the configuration.

Symptom Likely cause What to do
thales: failed to login Wrong credentials, wrong domain, or an unchanged initial password. Sign in to the console as the MinIO KMS user to confirm the password is final, and check the domain field.
A TLS or certificate error at startup MinIO KMS does not trust the CipherTrust Manager CA, or the endpoint host name does not match the certificate. Add the CA to tls.ca and use the host name from the certificate in server.
thales: missing CipherTrust Manager key name The key field is empty or missing. Set key to the name of the symmetric key.
failed to encrypt plaintext or failed to decrypt ciphertext The user lacks Encrypt or Decrypt on the key, or the key is deactivated. Check the key policy and the key state in the console.
Nodes start but do not form a cluster Nodes use different CipherTrust keys, so they derive different identities. Point every node at the same endpoint, domain, and key name.