AWS Secrets Manager

AWS Secrets Manager works as a key-value store for secrets like passwords, access tokens, and cryptographic keys. AWS encrypts these secrets with the AWS Key Management Service (AWS-KMS).

This tutorial shows how to set up a KES server that uses AWS Secrets Manager as a persistent key store protected by AWS-KMS:

AWS Secrets Manager

1. Create AWS Access/Secret Key Pair.

   • Go to the AWS console.

   • Create a new user.

     For details on adding a new AWS user, see the AWS docs.

   • Use the Programmatic access type to create a new access key / secret key pair.

2. Attach an AWS policy.

   Attach a policy or policies to the new user that grant access to the AWS Secrets Manager and the AWS-KMS.

   Your AWS IAM user must have the following permissions:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "Stmt1578498399136",
         "Action": [
           "secretsmanager:CreateSecret",
           "secretsmanager:DeleteSecret",
           "secretsmanager:GetSecretValue",
           "secretsmanager:ListSecrets"
         ],
         "Effect": "Allow",
         "Resource": "*"
       },
       {
         "Sid": "Stmt1578498562539",
         "Action": [
           "kms:Decrypt",
           "kms:DescribeKey",
           "kms:Encrypt"
         ],
         "Effect": "Allow",
         "Resource": "*"
       }
     ]
   }

   This example policy grants access to all KMS and SecretsManager resources. You can restrict access by specifying an AWS ARN as Resource instead of *.

   AWS has predefined policies (SecretsManagerReadWrite and AWSKeyManagementServicePowerUser). However, these grant more permissions than needed.

KES Server Setup

1. Generate the KES Server private key & certificate.

   First, generate a TLS private key and certificate for the KES server. A KES server is secure-by-default and can only be run with TLS. This example uses self-signed certificates for simplicity.

   The following command generates a new TLS private/public key pair and a certificate for the IP address 127.0.0.1 with the DNS name of localhost:

   $ kes identity new --ip "127.0.0.1" localhost
   
     Private key:  private.key
     Certificate:  public.crt
     Identity:     2e897f99a779cf5dd147e58de0fe55a494f546f4dcae8bc9e5426d2b5cd35680

   If you already have a TLS private key & certificate, such as from WebPKI or an internal CA, you can use those instead. Ensure the tls config section references the desired key & certificate.

2. Generate the client credentials.

   The following command generates a new TLS private/public key pair:

   $ kes identity new --key=client.key --cert=client.crt MyApp
   
     Private key:  client.key
     Certificate:  client.crt
     Identity:     02ef5321ca409dbc7b10e7e8ee44d1c3b91e4bf6e2198befdebee6312745267b

   The Identity is a unique fingerprint of the public key in client.crt that you can re-compute at any time:

   $ kes identity of client.crt
   
     Identity:  02ef5321ca409dbc7b10e7e8ee44d1c3b91e4bf6e2198befdebee6312745267b

3. Configure KES Server.

   Create the KES server configuration file: config.yml. The identity must match what is in the policy section of the client.crt identity.

   address: 0.0.0.0:7373 # Listen on all network interfaces on port 7373
   
   admin:
     identity: disabled  # We disable the admin identity since we don't need it in this guide 
      
   tls:
     key: private.key    # The KES server TLS private key
     cert: public.crt    # The KES server TLS certificate
      
   policy:
     my-app: 
       allow:
       - /v1/key/create/my-key*
       - /v1/key/generate/my-key*
       - /v1/key/decrypt/my-key*
       identities:
       - 02ef5321ca409dbc7b10e7e8ee44d1c3b91e4bf6e2198befdebee6312745267b # Use the identity of your client.crt
      
   keystore:
        aws:
          secretsmanager:
            endpoint: secretsmanager.us-east-2.amazonaws.com  # Use the SecretsManager in your region.
            region:   us-east-2                               # Use your region
            kmskey:   ""                                      # Your AWS-KMS master key (CMK) - optional.
            credentials:
              accesskey: "" # Your AWS Access Key
              secretkey: "" # Your AWS Secret Key

4. Start KES Server.

   kes server --config config.yml --auth off

   Linux Swap Protection

   In Linux environments, KES can use the mlock syscall to prevent the OS from writing in-memory data to disk (swapping). This prevents leaking sensitive data.

   Use the following command to allow KES to use the mlock syscall without running with root privileges:

   sudo setcap cap_ipc_lock=+ep $(readlink -f $(which kes))

   Start a KES server instance with memory protection:

   kes server --config config.yml --auth off --mlock

KES CLI Access

1. Set the MINIO_KES_SERVER Endpoint.

   This environment variable tells the KES CLI which server it should talk to.

   export MINIO_KES_SERVER=https://127.0.0.1:7373

2. Configure the client credentials.

   The following environment variables set the access credentials the KES CLI uses to talk to a KES server.

   export MINIO_KES_CERT_FILE=client.crt

   export MINIO_KES_KEY_FILE=client.key

3. Test access.

   Perform any API operation that is allowed based on the policy we assigned above.

   For example, to create a key:

   kes key create my-key-1

   Use the key to generate a new data encryption key:

   $ kes key dek my-key-1
   {
     plaintext : UGgcVBgyQYwxKzve7UJNV5x8aTiPJFoR+s828reNjh0=
     ciphertext: eyJhZWFkIjoiQUVTLTI1Ni1HQ00tSE1BQy1TSEEtMjU2IiwiaWQiOiIxMTc1ZjJjNDMyMjNjNjNmNjY1MDk5ZDExNmU3Yzc4NCIsIml2IjoiVHBtbHpWTDh5a2t4VVREV1RSTU5Tdz09Iiwibm9uY2UiOiJkeGl0R3A3bFB6S21rTE5HIiwiYnl0ZXMiOiJaaWdobEZrTUFuVVBWSG0wZDhSYUNBY3pnRWRsQzJqWFhCK1YxaWl2MXdnYjhBRytuTWx0Y3BGK0RtV1VoNkZaIn0=
   }

Using KES with a AIStor Server

AIStor Server requires KES to enable server-side data encryption.

See the KES for AIStor instruction guide for additional steps needed to use your new KES Server with a AIStor Server.

Configuration References

The following section describes the Key Encryption Service (KES) configuration settings to use AWS Secrets Manager and AWS Key Management System as the root KMS to store external keys, such as the keys used for Server-Side Encryption on a AIStor Server.

address: 0.0.0.0:7373
root: ${ROOT_IDENTITY}

tls:
  key: kes-server.key
  cert: kes-server.cert

api:
  /v1/ready:
    skip_auth: false
    timeout:   15s

policy:
  aistor-server:
    allow:
    - /v1/key/create/*
    - /v1/key/generate/*
    - /v1/key/decrypt/*
    - /v1/key/bulk/decrypt
    - /v1/key/list/*
    - /v1/status
    - /v1/metrics
    - /v1/log/audit
    - /v1/log/error
    deny:
    - /v1/key/generate/my-app-internal*
    - /v1/key/decrypt/my-app-internal*
    identities:
    - ${MINIO_IDENTITY}

  my-app:
    allow:
    - /v1/key/create/my-app*
    - /v1/key/generate/my-app*
    - /v1/key/decrypt/my-app*
    deny:
    - /v1/key/generate/my-app-internal*
    - /v1/key/decrypt/my-app-internal*
    identities:
    - df7281ca3fed4ef7d06297eb7cb9d590a4edc863b4425f4762bb2afaebfd3258
    - c0ecd5962eaf937422268b80a93dde4786dc9783fb2480ddea0f3e5fe471a731

keys:
  - name: "minio-encryption-key-alpha"
  - name: "minio-encryption-key-baker"
  - name: "minio-encryption-key-charlie"

cache:
  expiry:
    any: 5m0s
    unused: 20s
    offline: 0s

log:

  # Log error events to STDERR. Valid values are "on" or "off". 
  # Default is "on". 
  error: on

  # Log audit events to STDOUT. Valid values are "on" and "off". 
  # Default is "off". 
  audit: off

keystore:
  aws:
    # See: https://aws.amazon.com/secrets-manager
    secretsmanager:
      endpoint: secretsmanager.REGION.amazonaws
      region: REGION 
      kmskey: "" 
      credentials: 
        accesskey: "${AWS_ACCESS_KEY}" 
        secretkey: "${AWS_SECRET_KEY}" 
        token: ""