MinIO Kubernetes Plugin

Table of Contents

Overview
Installation
MinIO Kubernetes Plugin Syntax

Overview

Current Stable Version is 4.2.7

This reference documentation reflects 4.2.7 of the MinIO Kubernetes Operator and kubectl minio plugin.

The kubectl minio plugin brings native support for deploying MinIO tenants to Kubernetes clusters using the kubectl CLI. You can use kubectl minio to deploy a MinIO tenant with little to no interaction with YAML configuration files.

Kubernetes Orchestration with the MinIO Operator facilitates automated deployment of MinIO clusters.

Installing kubectl minio implies installing the MinIO Kubernetes Operator.

Installation

The MinIO Kubernetes Plugin requires Kubernetes 1.19.0 or later:

The following code downloads the latest stable version 4.2.7 of the MinIO Kubernetes Plugin and installs it to the system $PATH:

wget https://github.com/minio/operator/releases/download/v4.2.7/kubectl-minio_4.2.7_linux_amd64 -O kubectl-minio
chmod +x kubectl-minio
mv kubectl-minio /usr/local/bin/

You can access the plugin using the kubectl minio command. Run the following command to verify installation of the plugin:

kubectl minio version

MinIO Kubernetes Plugin Syntax

Create the MinIO Operator

kubectl minio init

Initializes the MinIO Operator. kubectl minio requires the operator for core functionality.

The command has the following syntax:

kubectl minio init [FLAGS]

The command supports the following arguments:

--image: The image to use for deploying the operator. Defaults to the latest release of the operator.

--namespace

The namespace into which to deploy the operator.

Defaults to minio-operator.

--cluster-domain: The domain name to use when configuring the DNS hostname of the operator. Defaults to cluster.local.

--namespace-to-watch

The namespace which the operator watches for MinIO tenants.

Defaults to "" or all namespaces.

--image-pull-secret

Secret key for use with pulling the --image.

The MinIO-hosted minio/k8s-operator image is not password protected. This option is only required for non-MinIO image sources which are password protected.

--console-image

The image to use when deploying the MinIO Console in Operator mode, where administrators can create and manage MinIO tenants using a Graphical User Interface.

Defaults to minio/console:v0.7.4.

--default-minio-image: The default minio image to use when creating a new MinIO tenant. Defaults to minio/minio:RELEASE.2021-06-07T21-40-51Z.

--default-console-image: The default MinIO Console image to use when creating a new MinIO tenant. Defaults to minio/console:v0.7.4.

--default-kes-image: The default kes image to use when creating a new MinIO tenant. Defaults to minio/kes:v0.14.0.

--output: Performs a dry run and outputs the generated YAML to STDOUT. Use this option to customize the YAML and apply it manually using kubectl apply -f <FILE>.

Delete the MinIO Operator

kubectl minio delete

Deletes the MinIO Operator along with all associated resources, including all MinIO Tenant instances in the watched namespace.

Warning

If the underlying Persistent Volumes (PV) were created with a reclaim policy of recycle or delete, deleting the MinIO Tenant results in complete loss of all objects stored on the tenant.

Ensure you have performed all due diligence in confirming the safety of any data on the MinIO Tenant prior to deletion.

The command has the following syntax:

kubectl minio delete [FLAGS]

The command accepts the following arguments:

--namespace

The namespace of the MinIO operator to delete.

Defaults to minio-operator.

Access the MinIO Operator Console

Command Requires MinIO Operator

Use the following command to validate that the operator is online and available prior to running this command:

kubectl get deployments -A --field-selector metadata.name=minio-operator

Issue the kubectl minio init command to initiate the operator if it is not already running in the Kubernetes cluster.

kubectl minio proxy

Create a temporary proxy to forward traffic from the local host machine to the MinIO Operator Console. The Operator Console provides a rich user interface for deploying and managing MinIO Tenants.

This command is an alternative to configuring Ingress to grant access to the Operator Console pods.

The command has the following syntax:

kubectl minio proxy

Create a MinIO Tenant

Command Requires MinIO Operator

Use the following command to validate that the operator is online and available prior to running this command:

kubectl get deployments -A --field-selector metadata.name=minio-operator

Issue the kubectl minio init command to initiate the operator if it is not already running in the Kubernetes cluster.

kubectl minio tenant create

Creates a MinIO Tenant with the following resources in the Kubernetes cluster. create always uses the latest stable version of the MinIO Server and MinIO Console.

The command has the following syntax:

kubectl minio tenant create TENANT_NAME FLAGS [ FLAGS ]

The following example creates a MinIO Tenant consisting of 4 MinIO servers with 8 drives each and a total capacity of 32Ti:

kubectl minio tenant create minio-tenant-1 \
  --servers          4                     \
  --volumes          8                     \
  --capacity         32Ti                  \
  --namespace        minio-tenant-1        \
  --storage-class local-storage

On success, the command returns the following:

The administrative username and password for the Tenant. Store these credentials in a secure location, such as a password protected key manager. MinIO does not show these credentials again.
The Service created for connecting to the MinIO Console. The Console supports administrative operations on the Tenant, such as configuring Identity and Access Management (IAM) and bucket configurations.
The Service created for connecting to the MinIO Tenant. Applications should use this service for performing operations against the MinIO Tenant.

The command supports the following arguments:

TENANT_NAME

Required

The name of the MinIO tenant which the command creates. The name must be unique in the namespace.

--servers

Required

The number of minio servers to deploy on the Kubernetes cluster.

Ensure that the specified number of --servers does not exceed the number of nodes in the Kubernetes cluster. MinIO strongly recommends sizing the cluster to have one node per MinIO server.

--volumes

Required

The number of volumes in the MinIO tenant. kubectl minio tenant create generates one Persistent Volume Claim (PVC) for each volume.

The number of volumes affects both the requested storage of each PVC and the number of PVC to associate to each MinIO Pod in the cluster:

The command kubectl minio divides the --capacity by the number of volumes to determine the amount of resources.requests.storage to set for each PVC.
kubectl minio determines the number of PVC to associate to each minio server by dividing --volumes by --servers.

The command generates each PVC with Pod-specific selectors, such that each Pod only uses PV that are locally-attached to the node running that Pod.

If the specified number of volumes exceeds the number of unbound PV available on the cluster, kubectl minio tenant create hangs and waits until the required PV exist.

--capacity

Required

The total capacity of the MinIO tenant. kubectl minio divides the capacity by the number of --volumes to determine the amount of resources.requests.storage to set for each Persistent Volume Claim (PVC).

If no Persistent Volumes (PV) can satisfy the requested storage, kubectl minio tenant create hangs and waits until the required storage exists.

--storage-class

Optional

The name of the Kubernetes Storage Class to use when creating Persistent Volume Claims (PVC) for the MinIO Tenant. The specified --storage-class must match the storage-class of the Persistent Volumes (PVs) to which the PVCs should bind.

MinIO strongly recommends creating a Storage Class that corresponds to locally-attached volumes on the host machines on which the Tenant deploys. This ensures each pod can use locally-attached storage for maximum performance and throughput. See the Deploy a MinIO Tenant tutorial for guidance on creating Storage Classes for supporting the MinIO Tenant.

Defaults to default.

--namespace

Optional

The namespace in which to create the MinIO Tenant and its associated resources.

MinIO supports exactly one MinIO Tenant per namespace. Create a unique namespace for each MinIO Tenant deployed into the cluster.

Defaults to minio.

--kes-config: The name of the Kubernetes Secret which contains the MinIO Key Encryption Service (KES) configuration. Required for enabling Server Side Encryption of objects (SSE-S3).

--output, --o

Outputs the generated YAML-formatted specification objects to STDOUT for further customization.

--output does not create the MinIO Tenant. Use kubectl apply -f <FILE> to manually create the MinIO tenant using the generated file.

Expand a MinIO Tenant

Command Requires MinIO Operator

Use the following command to validate that the operator is online and available prior to running this command:

kubectl get deployments -A --field-selector metadata.name=minio-operator

Issue the kubectl minio init command to initiate the operator if it is not already running in the Kubernetes cluster.

kubectl minio tenant expand

Extends the total capacity of a MinIO Tenant by adding a new Pool. A Pool consists of an independent set of pods running the MinIO Server and MinIO Console. The new pool uses the same Docker image for the MinIO Server and Console as the existing Tenant.

The command has the following syntax:

kubectl minio tenant expand TENANT_NAME --REQ_FLAGS [OPT_FLAGS]

The following example expands a MinIO Tenant with a Pool consisting of 4 MinIO servers with 8 drives each and a total additional capacity of 32Ti:

kubectl minio tenant expand minio-tenant-1 \
  --servers          4                     \
  --volumes          8                     \
  --capacity         32Ti                  \
  --namespace        minio-tenant-1        \
  --storage-class local-storage

The command supports the following arguments:

TENANT_NAME

Required

The name of the MinIO Tenant which the command expands.

--servers

Required

The number of minio servers to deploy in the new MinIO Tenant Pool.

Ensure that the specified number of --servers does not exceed the number of available nodes in the Kubernetes cluster.

--volumes

Required

The number of volumes in the new MinIO Tenant Pool. kubectl minio generates one Persistent Volume Claim (PVC) for each volume.

The number of volumes affects both the requested storage of each PVC and the number of PVC to associate to each MinIO Pod in the new Pool:

The command kubectl minio divides the --capacity by the number of volumes to determine the amount of resources.requests.storage to set for each PVC.
kubectl minio determines the number of PVC to associate to each minio server by dividing --volumes by --servers.

The command generates each PVC with Pod-specific selectors, such that each Pod only uses PV that are locally-attached to the node running that Pod.

If the specified number of volumes exceeds the number of unbound PV available in the cluster, kubectl minio tenant expand hangs and waits until the required PV exist.

--capacity

Required

The total capacity of the new MinIO Tenant Pool. kubectl minio divides the capacity by the number of --volumes to determine the amount of resources.requests.storage to set for each Persistent Volume Claim (PVC).

If the existing Persistent Volumes (PV) can satisfy the requested storage, kubectl minio tenant expand hangs and waits until the required storage exists.

--namespace

The namespace in which to create the new MinIO Tenant Pool. The namespace must match that of the MinIO Tenant being extended.

Defaults to minio.

--storage-class

Optional

The name of the Kubernetes Storage Class to use when creating Persistent Volume Claims (PVC) for the new MinIO Tenant Pool. The specified --storage-class must match the storage-class of the Persistent Volumes (PVs) to which the PVCs should bind.

MinIO strongly recommends creating a Storage Class that corresponds to locally-attached volumes on the host machines on which the Tenant deploys. This ensures each pod can use locally-attached storage for maximum performance and throughput. See the Deploy a MinIO Tenant tutorial for guidance on creating Storage Classes for supporting the MinIO Tenant.

--output

Outputs the generated YAML objects to STDOUT for further customization.

--output does not create the new MinIO Tenant Pool. Use kubectl apply -f <FILE> to manually create the MinIO tenant using the generated file.

Get MinIO Tenant Details

Command Requires MinIO Operator

Use the following command to validate that the operator is online and available prior to running this command:

kubectl get deployments -A --field-selector metadata.name=minio-operator

Issue the kubectl minio init command to initiate the operator if it is not already running in the Kubernetes cluster.

kubectl minio tenant info

Displays information on a MinIO Tenant, including but not limited to:

The total capacity of the Tenant
The version of MinIO server and MinIO Console running on the Tenant
The configuration of each Pool in the Tenant.

The command has the following syntax:

kubectl minio tenant info TENANT_NAME [ FLAGS ]

The following example retrieves the information of the MinIO Tenant minio-tenant-1 in the namespace minio-tenant-1.

kubectl minio tenant info minio-tenant-1 \
  --namespace minio-tenant-1

The command supports the following arguments:

TENANT_NAME

Required

The name of the MinIO Tenant for which the command returns the existing zones.

--namespace

Optional

The namespace in which to look for the MinIO Tenant.

Defaults to minio.

Upgrade MinIO Tenant

Command Requires MinIO Operator

Use the following command to validate that the operator is online and available prior to running this command:

kubectl get deployments -A --field-selector metadata.name=minio-operator

Issue the kubectl minio init command to initiate the operator if it is not already running in the Kubernetes cluster.

kubectl minio tenant upgrade

Upgrades the minio server Docker image used by the MinIO Tenant.

Important

MinIO upgrades the image used by all pods in the Tenant at once. This may result in downtime until the upgrade process completes.

The command has the following syntax:

kubectl minio tenant upgrade TENANT_NAME FLAGS [FLAGS]

The following example upgrades a MinIO Tenant to use the latest stable version of the MinIO server:

kubectl minio tenant upgrade minio-tenant-1 \
  --image  minio/minio

The command supports the following arguments:

TENANT_NAME

Required

The name of the MinIO Tenant which the command updates.

--image

Required

The Docker image to use for upgrading the MinIO Tenant.

--namespace

The namespace in which to look for the MinIO Tenant.

Defaults to minio.

--output

Outputs the generated YAML-formatted specification objects to STDOUT for further customization.

--output does not upgrade the MinIO Tenant. Use kubectl apply -f <FILE> to manually upgrade the MinIO tenant using the generated file.

Delete a MinIO Tenant

Command Requires MinIO Operator

Use the following command to validate that the operator is online and available prior to running this command:

kubectl get deployments -A --field-selector metadata.name=minio-operator

Issue the kubectl minio init command to initiate the operator if it is not already running in the Kubernetes cluster.

kubectl minio tenant delete

Deletes the MinIO Tenant and its associated resources.

The delete behavior of each Persistent Volume Claims (PVC) generated by the Tenant depends on the Reclaim Policy of its bound Persistent Volume (PV):

For recycle or delete policies, the command deletes the PVC.
For retain, the command retains the PVC.

Deletion of the underlying PV, whether automatic or manual, results in the loss of any objects stored on the MinIO Tenant. Perform all due diligence in ensuring the safety of stored data prior to deleting the tenant.

The command has the following syntax:

kubectl minio tenant delete TENANT_NAME FLAGS [ FLAGS ]

The command includes a confirmation prompt that requires explicit approval of the delete operation.

kubectl minio tenant delete --names TENANT_NAME [OPTIONAL_FLAGS]

The command supports the following arguments:

TENANT_NAME

Required

The name of the MinIO Tenant to delete.

--namespace

Optional

The namespace in which to look for the MinIO Tenant.

Defaults to minio.