Deploy a MinIO Tenant using the MinIO Plugin

Table of Contents

Prerequisites
Procedure

This procedure documents deploying a MinIO Tenant using the MinIO Kubernetes Plugin kubectl minio.

Kubernetes administrators who require more specific customization of Tenants prior to deployment can use this tutorial to create a validated yaml resource file for further modification.

The following procedure creates a MinIO tenant using the kubectl minio plugin. This procedure assumes the MinIO Operator is installed on the Kubernetes cluster. See Deploy MinIO Operator on Kubernetes for complete documentation on deploying the MinIO Operator.

Prerequisites

MinIO Operator 4.3.7

The MinIO Kubernetes plugin requires the MinIO Kubernetes Operator. This procedure assumes the latest stable Operator version 4.3.7.

See Deploy MinIO Operator on Kubernetes for complete documentation on deploying the MinIO Operator.

Kubernetes Version 1.19.0

Starting with v4.0.0, the MinIO Operator and MinIO Kubernetes Plugin require Kubernetes 1.19.0 and later. The Kubernetes infrastructure and the kubectl CLI tool must have the same version of 1.19.0+.

Locally Attached Drives

MinIO strongly recommends using locally attached drives on each node intended to support the MinIO Tenant. MinIO’s strict read-after-write and list-after-write consistency model requires local disk filesystems (xfs, ext4, etc.). MinIO also shows best performance with locally-attached drives.

MinIO automatically generates Persistent Volume Claims (PVC) as part of deploying a MinIO Tenant. The Operator generates one PVC for each volume in the tenant. For example, deploying a Tenant with 16 volumes requires 16 PV.

This procedure uses the MinIO DirectCSI driver to automatically provision Persistent Volumes from locally attached drives to support the generated PVC. See the DirectCSI Documentation for installation and configuration instructions.

For clusters which cannot deploy MinIO Direct CSI, use Local Persistent Volumes.

The following tabs provide example YAML objects for a local persistent volume and a supporting StorageClass:

The following YAML describes a Local Persistent Volume:

apiVersion: v1
kind: PersistentVolume
metadata:
   name: <PV-NAME>
spec:
   capacity:
      storage: 1Ti
   volumeMode: Filesystem
   accessModes:
   - ReadWriteOnce
   persistentVolumeReclaimPolicy: Retain
   storage-class: <STORAGE-CLASS>
   local:
      path: <PATH-TO-DISK>
   nodeAffinity:
      required:
         nodeSelectorTerms:
         - matchExpressions:
            - key: kubernetes.io/hostname
               operator: In
               values:
               - <NODE-NAME>

Replace values in brackets <VALUE> with the appropriate value for the local drive.

The following YAML describes a StorageClass that meets the requirements for a MinIO Tenant:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
   name: minio-local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

The storage class must have volumeBindingMode: WaitForFirstConsumer. Ensure all Persistent Volumes provisioned to support the MinIO Tenant use this storage class.

Procedure

1) Install the MinIO Plugin

The following code downloads the latest stable version 4.3.7 of the MinIO Kubernetes Plugin and installs it to the system $PATH:

wget https://github.com/minio/operator/releases/download/v4.3.7/kubectl-minio_4.3.7_linux_amd64 -O kubectl-minio
chmod +x kubectl-minio
mv kubectl-minio /usr/local/bin/

You can access the plugin using the kubectl minio command. Run the following command to verify installation of the plugin:

kubectl minio version

You can skip this step if the MinIO Plugin is installed on the host machine. Use kubectl minio version to check whether the plugin is already installed.

2) Create a Namespace for the MinIO Tenant

Use the kubectl create namespace command to create a namespace for the MinIO Tenant:

kubectl create namespace minio-tenant-1

MinIO supports exactly one Tenant per namespace.

3) Create the MinIO Tenant

Use the kubectl minio tenant create command to create the MinIO Tenant.

The following example creates a 4-node MinIO deployment with a total capacity of 16Ti across 16 drives.

kubectl minio tenant create minio-tenant-1       \
  --servers                 4                    \
  --volumes                 16                   \
  --capacity                16Ti                 \
  --storage-class           direct-csi-min-io    \
  --namespace               minio-tenant-1

The following table explains each argument specified to the command:

Argument	Description
`minio-tenant-1`	The name of the MinIO Tenant which the command creates.
`--servers`	The number of `minio` servers to deploy across the Kubernetes cluster.
`--volumes`	The number of volumes in the cluster. `kubectl minio` determines the number of volumes per server by dividing `volumes` by `servers`.
`--capacity`	The total capacity of the cluster. `kubectl minio` determines the capacity of each volume by dividing `capacity` by `volumes`.
`--storage-class`	The Kubernetes `StorageClass` to use when creating each PVC. This example uses the MinIO DirectCSI storage class.
`--namespace`	The Kubernetes namespace in which to deploy the MinIO Tenant.

On success, the command returns the following:

The administrative username and password for the Tenant. Store these credentials in a secure location, such as a password protected key manager. MinIO does not show these credentials again.
The Service created for connecting to the MinIO Console. The Console supports administrative operations on the Tenant, such as configuring Identity and Access Management (IAM) and bucket configurations.
The Service created for connecting to the MinIO Tenant. Applications should use this service for performing operations against the MinIO Tenant.

4) Configure Access to the Service

kubectl minio creates a service for the MinIO Tenant and MinIO Console. The output of kubectl minio tenant create includes the details for both services. You can also use kubectl get svc to retrieve the service name:

kubectl get svc --namespace minio-tenant-1

The command returns output similar to the following:

NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
minio                    ClusterIP   10.109.88.X     <none>        443/TCP             137m
minio-tenant-1-console   ClusterIP   10.97.87.X      <none>        9090/TCP,9443/TCP   129m
minio-tenant-1-hl        ClusterIP   None            <none>        9000/TCP            137m

The minio service corresponds to the MinIO Tenant service. Applications should use this service for performing operations against the MinIO Tenant.
The minio-tenant-1-console service corresponds to the MinIO Console. Administrators should use this service for accessing the MinIO Console and performing administrative operations on the MinIO Tenant.
The minio-tenant-1-hl corresponds to a headless service used to facilitate communication between Pods in the Tenant.

By default each service is visible only within the Kubernetes cluster. Applications deployed inside the cluster can access the services using the CLUSTER-IP. For applications external to the Kubernetes cluster, you must configure the appropriate network rules to expose access to the service. Kubernetes provides multiple options for configuring external access to services. See the Kubernetes documentation on Publishing Services (ServiceTypes) and Ingress for more complete information on configuring external access to services.

You can temporarily expose each service using the kubectl port-forward utility. Run the following examples to forward traffic from the local host running kubectl to the services running inside the Kubernetes cluster.

kubectl port-forward service/minio 443:443

kubectl port-forward service/minio-tenant-1-console 9443:9443