Deploy a MinIO Tenant using the MinIO Plugin

MinIO Operator 4.4.16

The MinIO Kubernetes plugin requires the MinIO Kubernetes Operator. This procedure assumes the latest stable Operator version 4.4.16.

See Deploy MinIO Operator on Kubernetes for complete documentation on deploying the MinIO Operator.

Kubernetes Version 1.19.0

Starting with v4.0.0, the MinIO Operator and MinIO Kubernetes Plugin require Kubernetes 1.19.0 and later. The Kubernetes infrastructure and the kubectl CLI tool must have the same version of 1.19.0+.

Locally Attached Drives

MinIO strongly recommends using locally attached drives on each node intended to support the MinIO Tenant. MinIO’s strict read-after-write and list-after-write consistency model requires local disk filesystems (xfs, ext4, etc.). MinIO also shows best performance with locally-attached drives.

MinIO automatically generates Persistent Volume Claims (PVC) as part of deploying a MinIO Tenant. The Operator generates one PVC for each volume in the tenant. For example, deploying a Tenant with 16 volumes requires 16 PV.

This procedure uses the MinIO DirectCSI driver to automatically provision Persistent Volumes from locally attached drives to support the generated PVC. See the DirectCSI Documentation for installation and configuration instructions.

For clusters which cannot deploy MinIO Direct CSI, use Local Persistent Volumes.

The following tabs provide example YAML objects for a local persistent volume and a supporting StorageClass:

apiVersion: v1
kind: PersistentVolume
metadata:
   name: <PV-NAME>
spec:
   capacity:
      storage: 1Ti
   volumeMode: Filesystem
   accessModes:
   - ReadWriteOnce
   persistentVolumeReclaimPolicy: Retain
   storage-class: <STORAGE-CLASS>
   local:
      path: <PATH-TO-DISK>
   nodeAffinity:
      required:
         nodeSelectorTerms:
         - matchExpressions:
            - key: kubernetes.io/hostname
               operator: In
               values:
               - <NODE-NAME>

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
   name: minio-local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

1) Install the MinIO Plugin

The following code downloads the latest stable version 4.4.16 of the MinIO Kubernetes Plugin and installs it to the system $PATH:

wget https://github.com/minio/operator/releases/download/v4.4.16/kubectl-minio_4.4.16_linux_amd64 -O kubectl-minio
chmod +x kubectl-minio
mv kubectl-minio /usr/local/bin/

You can access the plugin using the kubectl minio command. Run the following command to verify installation of the plugin:

kubectl minio version

You can skip this step if the MinIO Plugin is installed on the host machine. Use kubectl minio version to check whether the plugin is already installed.

2) Create a Namespace for the MinIO Tenant

Use the kubectl create namespace command to create a namespace for the MinIO Tenant:

kubectl create namespace minio-tenant-1

MinIO supports exactly one Tenant per namespace.

3) Create the MinIO Tenant

Use the kubectl minio tenant create command to create the MinIO Tenant.

The following example creates a 4-node MinIO deployment with a total capacity of 16Ti across 16 drives.

kubectl minio tenant create minio-tenant-1       \
  --servers                 4                    \
  --volumes                 16                   \
  --capacity                16Ti                 \
  --storage-class           direct-csi-min-io    \
  --namespace               minio-tenant-1

The following table explains each argument specified to the command:

On success, the command returns the following:

• The administrative username and password for the Tenant. Store these credentials in a secure location, such as a password protected key manager. MinIO does not show these credentials again.

• The Service created for connecting to the MinIO Console. The Console supports administrative operations on the Tenant, such as configuring Identity and Access Management (IAM) and bucket configurations.

• The Service created for connecting to the MinIO Tenant. Applications should use this service for performing operations against the MinIO Tenant.

4) Configure Access to the Service

kubectl minio creates a service for the MinIO Tenant and MinIO Console. The output of kubectl minio tenant create includes the details for both services. You can also use kubectl get svc to retrieve the service name:

kubectl get svc --namespace minio-tenant-1

The command returns output similar to the following:

NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
minio                    ClusterIP   10.109.88.X     <none>        443/TCP             137m
minio-tenant-1-console   ClusterIP   10.97.87.X      <none>        9090/TCP,9443/TCP   129m
minio-tenant-1-hl        ClusterIP   None            <none>        9000/TCP            137m

• The minio service corresponds to the MinIO Tenant service. Applications should use this service for performing operations against the MinIO Tenant.

• The minio-tenant-1-console service corresponds to the MinIO Console. Administrators should use this service for accessing the MinIO Console and performing administrative operations on the MinIO Tenant.

• The minio-tenant-1-hl corresponds to a headless service used to facilitate communication between Pods in the Tenant.

By default each service is visible only within the Kubernetes cluster. Applications deployed inside the cluster can access the services using the CLUSTER-IP. For applications external to the Kubernetes cluster, you must configure the appropriate network rules to expose access to the service. Kubernetes provides multiple options for configuring external access to services. See the Kubernetes documentation on Publishing Services (ServiceTypes) and Ingress for more complete information on configuring external access to services.

You can temporarily expose each service using the kubectl port-forward utility. Run the following examples to forward traffic from the local host running kubectl to the services running inside the Kubernetes cluster.

kubectl port-forward service/minio 443:443

kubectl port-forward service/minio-tenant-1-console 9443:9443