Azure Key Vault

This tutorial explains how to setup a KES Server to use Azure Key Vault as the persistent key store:

Azure Key Vault

Azure Key Vault is a managed KMS service that provides a secret store that can be used by KES.

As an external application, you must register KES in Azure Active Directory and have client credentials to store and access secrets in Azure Key Vault.

1. Active Directory Service

   • Navigate to Azure Active Directory and select App registrations.

     

     

   • Select New registration

2. Register KES App

   Give the application a name, such as KES-Demo, and register it. Once completed, Azure will show you some details about your newly registered application.

   

   Take note of the following fields to use later:

   • Application (client) ID
   • Directory (tenant) ID

   These fields contain universally unique identifiers (UUIDs), similar to c3b7badf-cd2b-4297-bece-4de5f2e575f6.

3. Create Client Secret

   • Select Add a certificate or secret
   • Assign a name, such as KES-Demo
   • Select an expiration

   Azure creates a new secret with the chosen description and expiry. This secret is required by KES to authenticate to Azure Key Vault.

   Make sure to copy the secret value. It may not be shown again.

   

4. Application Summary

   Navigate back to the application overview, and check that Azure shows that the application has one secret.

   

   You should have the following information:

   • Application (client) ID

     In our example images, c3b7badf-cd2b-4297-bece-4de5f2e575f6.

   • Directory (tenant) ID

     In our example images, 41a37d4e-b3c4-49f4-b330-1114fb0271c8.

   • The value of the newly created secret

     In our example images, -.j4XP6Sa7E39.KWn-SL~Dgbz~H-H-TPxT.

5. Add a Key Vault Policy

   Navigate to the Key Vault Access policies tab and select Add Access Policy to create a Key Vault policy.

   

   Define which Key Vault operations the KES Server can perform. Select the following five Secret permissions:

   • Get
   • List
   • Set
   • Delete
   • Purge

   

6. Assign Policy to Principal

   Select a principal or an authorized application.

   If the application itself is the principal, no authorized application is required. Alternatively, select a user or group as principal and select the newly registered KES Azure application as the authorized application.

   For this tutorial, we set the principal as the secret we added. Search for the name of the application (KES-Demo) or add the Application ID.

   

7. Policy Summary

   Azure shows a new access policy associated to our registered KES application.

   

   Make sure you Save before navigating elsewhere.

KES Server Setup

1. Generate KES Server Private Key & Certificate

   The KES server need both a TLS private key and a certificate. A KES server is secure-by-default and can only run with TLS. This tutorial uses self-signed certificates for simplicity.

   The following command generates a new TLS private key (private.key) and a self-signed X.509 certificate (public.crt) issued for the IP 127.0.0.1 and DNS name localhost:

   $ kes identity new --ip "127.0.0.1" localhost
   
     Private key:  private.key
     Certificate:  public.crt
     Identity:     2e897f99a779cf5dd147e58de0fe55a494f546f4dcae8bc9e5426d2b5cd35680

   If you already have a TLS private key & certificate from WebPKI or an internal CA, you can use them instead. Remember to adjust the tls config section.

2. Generate Client Credentials

   Use the following command to generate a new TLS private/public key pair for the client application to use to access the KES Server:

   $ kes identity new --key=client.key --cert=client.crt MyApp
   
     Private key:  client.key
     Certificate:  client.crt
     Identity:     02ef5321ca409dbc7b10e7e8ee44d1c3b91e4bf6e2198befdebee6312745267b

   The identity 02ef5321ca409dbc7b10e7e8ee44d1c3b91e4bf6e2198befdebee6312745267b is a unique fingerprint of the public key in client.crt. You can re-compute the fingerprint at anytime:

   $ kes identity of client.crt
   
     Identity:  02ef5321ca409dbc7b10e7e8ee44d1c3b91e4bf6e2198befdebee6312745267b

3. Configure the KES Server

   Create the KES server configuration file: config.yml. Make sure that the identity in the policy section matches your client.crt identity.

   address: 0.0.0.0:7373 # Listen on all network interfaces on port 7373
   
   admin:
     identity: disabled  # We disable the admin identity since we don't need it in this guide 
      
   tls:
     key: private.key    # The KES server TLS private key
     cert: public.crt    # The KES server TLS certificate
      
   policy:
     my-app: 
       allow:
       - /v1/key/create/my-key*
       - /v1/key/generate/my-key*
       - /v1/key/decrypt/my-key*
       identities:
       - 02ef5321ca409dbc7b10e7e8ee44d1c3b91e4bf6e2198befdebee6312745267b # Use the identity of your client.crt
      
   keystore:
        azure:
          keyvault:
            endpoint: "https://kes-test-1.vault.azure.net"    # Use your KeyVault instance endpoint.
            credentials:
              tenant_id: ""      # The ID of the tenant the client belongs to - e.g: "41a37d4e-b3c4-49f4-b330-1114fb0271c8".
              client_id: ""      # The ID of the client                       - e.g: "c3b7badf-cd2b-4297-bece-4de5f2e575f6".
              client_secret: ""  # The value of the client secret             - e.g: "-.j4XP6Sa7E39.KWn-SL~Dgbz~H-H-TPxT".

4. Start the KES Server

   Linux

   kes server --config config.yml --auth off

   Linux Swap Protection

   In Linux environments, KES can use the mlock syscall to prevent the OS from writing in-memory data to disk (swapping). This prevents leaking sensitive data.

   Use the following command to allow KES to use the mlock syscall without running with root privileges:

   sudo setcap cap_ipc_lock=+ep $(readlink -f $(which kes))

   Start a KES server instance with memory protection:

   kes server --config config.yml --auth off --mlock

KES CLI Access

1. Set the MINIO_KES_SERVER Endpoint

   The following environment variable specifies the server the KES CLI should talk to:

   export MINIO_KES_SERVER=https://127.0.0.1:7373

2. Define the Client Credentials

   The following environment variables set the access credentials the client uses to talk to a KES server:

   export MINIO_KES_CERT_FILE=client.crt

   export MINIO_KES_KEY_FILE=client.key

3. Test the Configuration

   Perform any API operation allowed by the policy we assigned above.

   For example, create a key:

   kes key create my-key-1

   Use the key to generate a new data encryption key:

   kes key dek my-key-1

   The result resembles the following:

   {
     plaintext : UGgcVBgyQYwxKzve7UJNV5x8aTiPJFoR+s828reNjh0=
     ciphertext: eyJhZWFkIjoiQUVTLTI1Ni1HQ00tSE1BQy1TSEEtMjU2IiwiaWQiOiIxMTc1ZjJjNDMyMjNjNjNmNjY1MDk5ZDExNmU3Yzc4NCIsIml2IjoiVHBtbHpWTDh5a2t4VVREV1RSTU5Tdz09Iiwibm9uY2UiOiJkeGl0R3A3bFB6S21rTE5HIiwiYnl0ZXMiOiJaaWdobEZrTUFuVVBWSG0wZDhSYUNBY3pnRWRsQzJqWFhCK1YxaWl2MXdnYjhBRytuTWx0Y3BGK0RtV1VoNkZaIn0=
   }

Using KES with a AIStor Server

AIStor Server requires KES to enable server-side data encryption.

See the KES for AIStor instruction guide for additional steps needed to use your new KES Server with a AIStor Server.

Configuration References

The following section describes the Key Encryption Service (KES) configuration settings to use Azure Key Vault as the root KMS to store external keys, such as the keys used for Server-Side Encryption on a AIStor Server.

address: 0.0.0.0:7373
root: ${ROOT_IDENTITY}

tls:
  key: kes-server.key
  cert: kes-server.cert

api:
  /v1/ready:
    skip_auth: false
    timeout:   15s

policy:
  minio-server:
    allow:
    - /v1/key/create/*
    - /v1/key/generate/*
    - /v1/key/decrypt/*
    - /v1/key/bulk/decrypt
    - /v1/key/list/*
    - /v1/status
    - /v1/metrics
    - /v1/log/audit
    - /v1/log/error
    deny:
    - /v1/key/generate/my-app-internal*
    - /v1/key/decrypt/my-app-internal*
    identities:
    - ${MINIO_IDENTITY}

  my-app:
    allow:
    - /v1/key/create/my-app*
    - /v1/key/generate/my-app*
    - /v1/key/decrypt/my-app*
    deny:
    - /v1/key/generate/my-app-internal*
    - /v1/key/decrypt/my-app-internal*
    identities:
    - df7281ca3fed4ef7d06297eb7cb9d590a4edc863b4425f4762bb2afaebfd3258
    - c0ecd5962eaf937422268b80a93dde4786dc9783fb2480ddea0f3e5fe471a731

keys:
  - name: "minio-encryption-key-alpha"
  - name: "minio-encryption-key-baker"
  - name: "minio-encryption-key-charlie"

cache:
  expiry:
    any: 5m0s
    unused: 20s
    offline: 0s

log:

  # Log error events to STDERR. Valid values are "on" or "off". 
  # Default is "on".
  error: on

  # Log audit events to STDOUT. Valid values are "on" and "off". 
  # Default is "off".
  audit: off

keystore:
  azure:
    # See: https://azure.microsoft.com/services/key-vault
    keyvault:
      endpoint: "" 
      credentials:
        tenant_id: "" 
        client_id: "" 
        client_secret: "" 
      managed_identity:
        client_id: "" 

References

• Server API Doc
• Go SDK Doc