Troubleshooting

This page covers common issues and resolution steps for MinIO KMS deployments.

Startup failures

No TLS configuration provided

kms: no TLS configuration provided

MinIO KMS requires TLS for all connections. Verify that the TLS certificate and key paths are correct and that the files are readable by the MinIO KMS process.

No HSM provided

kms: no HSM provided

MinIO KMS requires an HSM (Hardware Security Module) backend. Configure either a software HSM or an external HSM provider such as HashiCorp Vault. See the HSM Key Management documentation.

No server certificate provided

kms: no server certificate provided

The TLS certificate file is missing or the path is incorrect. Verify the certificate path in the startup configuration. See Certificate Management for certificate setup.

Certificate issues

Certificate expired

Clients receive TLS handshake errors when connecting to a MinIO KMS node with an expired certificate.

Check certificate expiry:

openssl s_client -connect MINKMS_HOST:7373 </dev/null 2>/dev/null | openssl x509 -noout -enddate

MinIO KMS automatically reloads certificates when the underlying files change. Replace the certificate files and MinIO KMS picks up the new certificates without a restart. See Certificate Management for rotation procedures.

Certificate not trusted

tls: certificate signed by unknown authority

The client does not trust the CA that signed the MinIO KMS certificate. Add the CA certificate to the client’s trust store. For MinIO AIStor connecting to MinIO KMS, place the CA in the MinIO certs directory.

Cluster issues

Frequent leader elections

If the consens_elections metric increases rapidly, the cluster is experiencing instability. Common causes:

  • Network latency between nodes: verify internode round-trip time is below 10 ms
  • Resource starvation: check CPU and memory on all nodes. A node under heavy GC pressure may miss heartbeat deadlines
  • Uneven node count: MinIO KMS clusters should have an odd number of nodes (3, 5, 7) for reliable leader election

Check the consensus metrics:

curl -s --cacert ca.crt --cert client.crt --key client.key https://MINKMS_HOST:7373/v1/health/metrics | grep consens

Node not joining cluster

If a new node fails to join an existing cluster:

  • Verify the node can reach all existing cluster members on port 7373
  • Verify the TLS certificates include the correct SANs for the new node
  • Check that the new node uses the same enclave configuration as existing nodes

API errors

Key not found

Clients receive an error when attempting to use a key that does not exist.

Verify the key exists:

mc admin kms key status ALIAS KEY_NAME

If the key was created on a different enclave, verify the client is authenticating against the correct enclave.

Permission denied

The client identity does not have the required policy permissions.

Check the identity’s assigned policies:

mc admin kms identity info ALIAS IDENTITY

Verify the policy includes the required operations (for example, /v1/key/create/*, /v1/key/decrypt/*).

Connectivity

Health check verification

Verify MinIO KMS is running and responsive:

# Liveness (no TLS client auth required)
curl -k https://MINKMS_HOST:7373/v1/health/live

# Readiness
curl -s --cacert ca.crt --cert client.crt --key client.key https://MINKMS_HOST:7373/v1/health/ready

A 200 OK response confirms the node is healthy.

MinIO AIStor cannot reach MinIO KMS

If MinIO AIStor logs show KMS connectivity errors:

  1. Verify MinIO KMS is running: check the health endpoint from the MinIO node
  2. Verify the MinIO KMS CA certificate is in the MinIO AIStor trust store
  3. Verify the MinIO AIStor KMS configuration points to the correct endpoint
  4. Check for firewall rules blocking port 7373