Quick Start

Download the binary

curl -LO https://dl.min.io/memkv/release/linux-amd64/memkv
chmod +x memkv
sudo mv memkv /usr/local/bin/
memkv --version

linux-arm64 artefacts are also published; swap the URL accordingly.

Install the NIXL plugin

Download the prebuilt plugin and drop it into the NIXL plugin directory:

sudo mkdir -p /opt/nvidia/nvda_nixl/lib/plugins
sudo curl -L \
  https://dl.minio.io/aistor/memkv/release/linux-amd64/libplugin_MEMKV.so \
  -o /opt/nvidia/nvda_nixl/lib/plugins/libplugin_MEMKV.so

Override the install location with NIXL_PLUGIN_DIR=/custom/path. Supported NIXL memory types: DRAM_SEG, VRAM_SEG (via nvidia-peermem).

If you installed MemKV via the .deb or .rpm package instead of the raw binary, libplugin_MEMKV.so is already at /usr/lib/memkv/libplugin_MEMKV.so — symlink or copy it into the NIXL plugin directory.

Provision a license

Request a license (Free or Enterprise) at min.io/pricing and save the JWT under /etc/memkv/:

sudo install -m 0600 /path/to/minio.license /etc/memkv/minio.license

The server picks it up automatically. Override path with MEMKV_LICENSE=/some/other/path if needed.

Configure the client

The plugin reads memkv-client settings from a yaml file pointed at by MEMKV_CONFIG. Create a minimal one:

sudo mkdir -p /etc/memkv
sudo tee /etc/memkv/client.yaml > /dev/null <<EOF
servers:
  - server-0.memkv.example.com:9900
  - server-1.memkv.example.com:9900
rdma_devices: [mlx5_0, mlx5_1]
transport: auto       # auto picks RDMA when rdma_devices is set, TCP otherwise
auth_key: REPLACE_WITH_SERVERS_AUTH_KEY
license: /etc/memkv/minio.license
EOF
export MEMKV_CONFIG=/etc/memkv/client.yaml

The auth_key must match what memkv setup generates on the servers — read it back from /etc/memkv/config.yaml (the network.auth_key field) and paste it here, or set the matching MEMKV_AUTH_KEY env var on both sides. Anything in the yaml can be overridden at runtime via the corresponding MEMKV_* env var (MEMKV_SERVERS, MEMKV_RDMA_DEVICES, MEMKV_LICENSE, MEMKV_AUTH_KEY, …). Full schema and precedence: Configuration → Client.

Initialize drives

memkv setup writes the generated YAML to stdout — redirect it to your config path:

sudo memkv setup \
  --drives /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 \
  --rdma mlx5_0 \
  | sudo tee /etc/memkv/config.yaml

See the CLI reference for the full flag list.

Start the server

sudo memkv start --config /etc/memkv/config.yaml

Verify the server is healthy:

curl http://localhost:9901/v1/health
# {"status":"healthy","rdma_active":true,"drives_online":3,"drives_offline":0,"drives_total":3}

Generate an auth key and write the server config

sudo mkdir -p /etc/memkv /var/lib/memkv
openssl rand -hex 32 | sudo tee /etc/memkv/auth_key > /dev/null
sudo chmod 600 /etc/memkv/auth_key
AUTH=$(sudo cat /etc/memkv/auth_key)
sudo tee /etc/memkv/config.yaml > /dev/null <<EOF
network:
  address: 127.0.0.1:9900         # data plane; admin auto-binds at
                                  # port + 1 (9901)
  auth_key: "$AUTH"
memory:
  block_size: 2 MiB
  max_size: 1 GiB
storage:
  mode: file
  block_size: 4 MiB
  drives:
    - media: /var/lib/memkv/drive0.dat
      max_size: 16 GiB
EOF

The shared secret is persisted to /etc/memkv/auth_key so step 3 can read it back even from a fresh shell.

Start the server

Save the license JWT to /etc/memkv/minio.license (the server auto-discovers it there) and start:

sudo install -m 0600 /path/to/minio.license /etc/memkv/minio.license
sudo memkv start --config /etc/memkv/config.yaml

The backing file is fully preallocated on first start (fallocate on Linux, fcntl(F_PREALLOCATE) on macOS) — first run is slower but there are no sparse-file surprises later.

Verify health (on Linux the response includes drive/RDMA fields; macOS returns the short form):

curl http://127.0.0.1:9901/v1/health
# macOS: {"status":"healthy"}
# Linux: {"status":"healthy","rdma_active":false,"drives_online":1,"drives_offline":0,"drives_total":1}

Point the inference client at it

Both the NIXL plugin and the LD_PRELOAD shim read settings from MEMKV_CONFIG. Force transport: tcp so the client doesn't try RDMA at all (macOS defaults to tcp automatically; Linux defaults to auto and would attempt RDMA first):

AUTH=$(sudo cat /etc/memkv/auth_key)
cat > "$HOME/memkv-client.yaml" <<EOF
servers:
  - 127.0.0.1:9900
transport: tcp
auth_key: "$AUTH"
license: /etc/memkv/minio.license
EOF
export MEMKV_CONFIG="$HOME/memkv-client.yaml"

Now any process that loads the NIXL plugin or the LD_PRELOAD shim routes context-cache traffic to the local MemKV server over TCP.

Pre-create raw block PVs for the NVMe drives

MemKV writes its JBOF superblock and extents directly to raw block devices. The chart's default drives.mode: blockDevice claims one PVC per drive per replica, bound to operator-supplied local PVs. The bound PV's nodeAffinity pins the Pod to the right host automatically.

Edit deploy/helm/memkv/examples/local-pv-example.yaml to match your nodes and drives, then apply:

kubectl apply -f deploy/helm/memkv/examples/local-pv-example.yaml

Create the license Secret

Request a license (Free or Enterprise) at min.io/pricing, save the JWT to minio.license, then load it into the cluster:

kubectl create namespace memkv
kubectl -n memkv create secret generic memkv-license \
    --from-file=license=/path/to/minio.license

Install the chart

helm install memkv deploy/helm/memkv \
    --namespace memkv \
    --set replicaCount=2 \
    --set drives.blockDevice.count=12 \
    --set drives.blockDevice.storageClass=memkv-local \
    --set license.existingSecret=memkv-license \
    --set config.memory.maxSize="64 GiB" \
    --set hugepages.amount=64Gi

helm install memkv deploy/helm/memkv \
    --namespace memkv \
    --set drives.mode=hostDev \
    --set 'drives.hostDev.devices={/dev/nvme0n1,/dev/nvme1n1}' \
    --set 'nodeSelector.memkv\.minio\.io/storage=true' \
    --set license.existingSecret=memkv-license

helm install memkv deploy/helm/memkv \
    --namespace memkv --create-namespace \
    -f deploy/helm/memkv/ci/ci-values.yaml

Verify the deployment

kubectl -n memkv get pods -o wide
kubectl -n memkv get pvc -l app.kubernetes.io/instance=memkv

Each Pod should land on a different node, and every PVC should be Bound. Check the admin endpoint:

kubectl -n memkv port-forward svc/memkv-admin 9901:9901
curl http://127.0.0.1:9901/v1/health
# {"status":"healthy","rdma_active":true,"drives_online":12,"drives_offline":0,"drives_total":12}