Table of Contents
This procedure documents deploying a MinIO Tenant using the
MinIO Kubernetes Plugin kubectl minio
.
Kubernetes administrators who require more specific customization of
Tenants prior to deployment can use this tutorial to create a validated
yaml
resource file for further modification.
The following procedure creates a MinIO tenant using the
kubectl minio
plugin. This procedure assumes the
MinIO Operator is installed on the Kubernetes cluster. See
Deploy MinIO Operator on Kubernetes for complete documentation on deploying the
MinIO Operator.
The MinIO Kubernetes plugin requires the MinIO Kubernetes Operator. This procedure assumes the latest stable Operator version 4.4.16.
See Deploy MinIO Operator on Kubernetes for complete documentation on deploying the MinIO Operator.
Starting with v4.0.0, the MinIO Operator and MinIO Kubernetes Plugin require
Kubernetes 1.19.0 and later. The Kubernetes infrastructure and the
kubectl
CLI tool must have the same version of 1.19.0+.
MinIO strongly recommends using locally attached drives on each node intended to support the MinIO Tenant. MinIO’s strict read-after-write and list-after-write consistency model requires local disk filesystems (xfs, ext4, etc.). MinIO also shows best performance with locally-attached drives.
MinIO automatically generates Persistent Volume Claims (PVC) as part of
deploying a MinIO Tenant. The Operator generates one PVC for each volume in the
tenant. For example, deploying a Tenant with 16 volumes requires 16 PV
.
This procedure uses the MinIO DirectCSI driver to automatically provision Persistent Volumes from locally attached drives to support the generated PVC. See the DirectCSI Documentation for installation and configuration instructions.
For clusters which cannot deploy MinIO Direct CSI, use Local Persistent Volumes.
The following tabs provide example YAML objects for a local persistent volume and a supporting StorageClass:
The following YAML describes a Local Persistent Volume:
apiVersion: v1
kind: PersistentVolume
metadata:
name: <PV-NAME>
spec:
capacity:
storage: 1Ti
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storage-class: <STORAGE-CLASS>
local:
path: <PATH-TO-DISK>
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- <NODE-NAME>
Replace values in brackets <VALUE>
with the appropriate
value for the local drive.
The following YAML describes a StorageClass that meets the requirements for a MinIO Tenant:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: minio-local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
The storage class must have volumeBindingMode: WaitForFirstConsumer
.
Ensure all Persistent Volumes provisioned to support the MinIO Tenant
use this storage class.
The following code downloads the latest stable version 4.4.16
of the MinIO Kubernetes Plugin and installs it to the system $PATH
:
wget https://github.com/minio/operator/releases/download/v4.4.16/kubectl-minio_4.4.16_linux_amd64 -O kubectl-minio
chmod +x kubectl-minio
mv kubectl-minio /usr/local/bin/
You can access the plugin using the kubectl minio
command. Run
the following command to verify installation of the plugin:
kubectl minio version
You can skip this step if the MinIO Plugin is installed on the host machine.
Use kubectl minio version
to check whether the plugin
is already installed.
Use the kubectl create namespace
command to create a namespace for
the MinIO Tenant:
kubectl create namespace minio-tenant-1
MinIO supports exactly one Tenant per namespace.
Use the kubectl minio tenant create
command to create the MinIO
Tenant.
The following example creates a 4-node MinIO deployment with a total capacity of 16Ti across 16 drives.
kubectl minio tenant create minio-tenant-1 \
--servers 4 \
--volumes 16 \
--capacity 16Ti \
--storage-class direct-csi-min-io \
--namespace minio-tenant-1
The following table explains each argument specified to the command:
Argument |
Description |
---|---|
The name of the MinIO Tenant which the command creates. |
|
The number of |
|
The number of volumes in the cluster. |
|
The total capacity of the cluster. |
|
The Kubernetes |
|
The Kubernetes namespace in which to deploy the MinIO Tenant. |
On success, the command returns the following:
The administrative username and password for the Tenant. Store these credentials in a secure location, such as a password protected key manager. MinIO does not show these credentials again.
The Service created for connecting to the MinIO Console. The Console supports administrative operations on the Tenant, such as configuring Identity and Access Management (IAM) and bucket configurations.
The Service created for connecting to the MinIO Tenant. Applications should use this service for performing operations against the MinIO Tenant.
kubectl minio
creates a service for the MinIO Tenant and MinIO Console.
The output of kubectl minio tenant create
includes the details for
both services. You can also use kubectl get svc
to retrieve the service
name:
kubectl get svc --namespace minio-tenant-1
The command returns output similar to the following:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
minio ClusterIP 10.109.88.X <none> 443/TCP 137m
minio-tenant-1-console ClusterIP 10.97.87.X <none> 9090/TCP,9443/TCP 129m
minio-tenant-1-hl ClusterIP None <none> 9000/TCP 137m
The minio
service corresponds to the MinIO Tenant service. Applications
should use this service for performing operations against the MinIO Tenant.
The minio-tenant-1-console
service corresponds to the MinIO Console.
Administrators should use this service for accessing the MinIO Console and
performing administrative operations on the MinIO Tenant.
The minio-tenant-1-hl
corresponds to a headless service used to
facilitate communication between Pods in the Tenant.
By default each service is visible only within the Kubernetes cluster.
Applications deployed inside the cluster can access the services using the
CLUSTER-IP
. For applications external to the Kubernetes cluster,
you must configure the appropriate network rules to expose access to the
service. Kubernetes provides multiple options for configuring external access
to services. See the Kubernetes documentation on
Publishing Services (ServiceTypes)
and Ingress
for more complete information on configuring external access to services.
You can temporarily expose each service using the
kubectl port-forward
utility. Run the following examples to forward
traffic from the local host running kubectl
to the services running inside
the Kubernetes cluster.
kubectl port-forward service/minio 443:443
kubectl port-forward service/minio-tenant-1-console 9443:9443