Deploy a MinIO Tenant

MinIO Operator 4.2.7

The Operator Console requires the MinIO Kubernetes Operator. This procedure assumes the latest stable Operator version 4.2.7.

See Deploy MinIO Operator on Kubernetes for complete documentation on deploying the MinIO Operator.

MinIO Kubernetes Plugin 4.2.7

Run the following commands to install the MinIO Operator and Plugin using the Kubernetes krew plugin manager:

kubectl krew update
kubectl krew install minio

See the krew installation documentation for specific instructions.

You can also download the kubectl-minio plugin directly and install it to your system PATH. The following code downloads the latest stable version 4.2.7 of the MinIO Kubernetes Plugin and installs it to the system $PATH:

wget https://github.com/minio/operator/releases/download/v4.2.7/kubectl-minio_4.2.7_linux_amd64 -O kubectl-minio
chmod +x kubectl-minio
mv kubectl-minio /usr/local/bin/

Run the following command to verify installation of the plugin:

kubectl minio version

The output should display the Operator version as 4.2.7.

Kubernetes Version 1.19.0

Starting with v4.0.0, the MinIO Operator requires Kubernetes 1.19.0 and later. The Kubernetes infrastructure and the kubectl CLI tool must have the same version of 1.19.0+.

This procedure assumes the host machine has kubectl installed and configured with access to the target Kubernetes cluster. The host machine must have access to a web browser application.

Locally Attached Drives

MinIO strongly recommends using locally attached drives on each node intended to support the MinIO Tenant. MinIO’s strict read-after-write and list-after-write consistency model requires local disk filesystems (xfs, ext4, etc.). MinIO also shows best performance with locally-attached drives.

MinIO automatically generates Persistent Volume Claims (PVC) as part of deploying a MinIO Tenant. The Operator generates one PVC for each volume in the tenant plus two PVC to support collecting Tenant Metrics and logs. For example, deploying a Tenant with 16 volumes requires 18 (16 + 2) PV.

This procedure uses the MinIO DirectCSI driver to automatically provision Persistent Volumes from locally attached drives to support the generated PVC. See the DirectCSI Documentation for installation and configuration instructions.

For clusters which cannot deploy MinIO Direct CSI, Local Persistent Volumes.

The following tabs provide example YAML objects for a local persistent volume and a supporting StorageClass:

apiVersion: v1
kind: PersistentVolume
metadata:
   name: <PV-NAME>
spec:
   capacity:
      storage: 1Ti
   volumeMode: Filesystem
   accessModes:
   - ReadWriteOnce
   persistentVolumeReclaimPolicy: Retain
   storage-class: <STORAGE-CLASS>
   local:
      path: <PATH-TO-DISK>
   nodeAffinity:
      required:
         nodeSelectorTerms:
         - matchExpressions:
            - key: kubernetes.io/hostname
               operator: In
               values:
               - <NODE-NAME>

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
   name: minio-local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

1) Access the MinIO Operator Console

Use the kubectl minio proxy command to temporarily forward traffic between the local host machine and the MinIO Operator Console:

kubectl minio proxy

The command returns output similar to the following:

Starting port forward of the Console UI.

To connect open a browser and go to http://localhost:9090

Current JWT to login: TOKEN

Open your browser to the specified URL and enter the JWT Token into the login page. You should see the Tenants page:

Click the + Create Tenant to start creating a MinIO Tenant.

2) Complete the Name Tenant Step

The i. Name Tenant step contains configuration settings related to the Tenant Name, Namespace, and Storage Class.

The specified Namespace must not contain any existing MinIO Tenants. Consider creating a new Namespace for the MinIO Tenant. You can create the namespace through the UI by entering the desired name and clicking the + icon.

This procedure assumes using the DirectCSI storage class direct-csi-min-io. See the DirectCSI Documentation for installation and configuration instructions.

The Advanced Mode toggle enables additional configuration options for the MinIO Tenant. This procedure provides a high level description of each of the advanced configuration sections.

Click Next to proceed.

3) Complete the Configure Step

The ii. Configure step contains configuration settings related to the MinIO Tenant. This step is only visible if you enabled Advanced Mode in step i. Name Tenant.

4) Complete the Pod Affinity Step

The iii. Pod Affinity step contains configuration settings related to scheduling MinIO Tenant Pods. This step is only visible if you enabled Advanced Mode in step i. Name Tenant.

Select the type of affinity rules you want to apply to the MinIO Tenant. The default Pod Anti-Affinity ensures that no two MinIO pods deploy to the same worker node.

5) Complete the Identity Provider Step

The iv. Identity Provider step contains configuration settings related to MinIO Identity and Access Management. This step is only visible if you enabled Advanced Mode in step i. Name Tenant.

The default Built-In provides configuration settings for the MinIO internal identity provider. See MinIO Internal IDP for more complete documentation.

You can also configure either an OpenID Connect or Active Directory service as an external identity manager. See the linked documentation for more information on configuring MinIO for external identity management.

6) Complete the Security Step

The v. Security step contains configuration settings related to MinIO Transport Layer Security (TLS). This step is only visible if you enabled Advanced Mode in step i. Name Tenant.

The MinIO Operator automatically generates TLS certificates using the Kubernetes certificates.k8s.io TLS certificate management API. You can disable this behavior by toggling Enable AutoCert.

You can provide one or more Custom Certificates for use by the MinIO Tenant. MinIO supports Server Name Indication (SNI) support for selecting which TLS certificate to respond with based on the hostname specified in a client request. The Operator automatically distributes the specified certificates to every server pod in the tenant.

Disabling AutoCert and specifying no Custom Certificates deploys the MinIO Tenant without TLS. Consider the security risks of allowing unsecured traffic before deploying Tenants without TLS.

7) Complete the Encryption Step

The vi. Encryption step contains configuration settings related to MinIO Server-Side Object Encryption. This step is only visible if you enabled Advanced Mode in step i. Name Tenant.

The Operator Console supports the following external Key Management Systems (KMS):

• Hashicorp Vault

• Thales CipherTrust (formerly Gemalto KeySecure)

• AWS KMS

• GCP Secrets Manager

For more complete documentation on the required fields, see MinIO KES Guides.

8) Complete the Tenant Size Step

The vii. Tenant Size step contains configuration settings related to the MinIO server pods deployed as part of the Tenant.

• The Resource Allocation section displays the resulting compute configuration.

• The Erasure Code Configuration section displays the resulting erasure code configuration.

You can also use the MinIO Erasure Code Calculator to help guide configuring the MinIO Tenant.

9) Complete the Preview Configuration Step

The viii. Preview Configuration step displays a summary of the Tenant configuration.

Click Create to begin the Tenant creation process. You can return to any previous section to modify the Tenant configuration before proceeding.

After clicking Create, the Operator Console displays the root credentials for the MinIO Tenant.

Download and copy the credentials to a secure location. The Operator never displays these credentials again.

10) View Tenant Details

You can monitor the Tenant creation process from the Tenants view. The State column updates throughout the deployment process.

Tenant deployment can take several minutes to complete. Once the State reads as Initialized, click the Tenant to view its details.

Each tab provides additional details or configuration options for the MinIO Tenant.

• METRICS - Displays metrics collected from the MinIO Tenant.

• SECURITY - Provides TLS-related configuration options.

• POOLS - Supports expanding the tenant by adding more Server Pools.

• LICENSE - Enter your SUBNET license.

11) Connect to the Tenant

The MinIO Operator creates services for the MinIO Tenant. Use the kubectl get svc -n NAMESPACE command to review the deployed services:

kubectl get svc -n minio-tenant-1

NAME                               TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
minio                              LoadBalancer   10.97.114.60     <pending>     443:30979/TCP    2d3h
minio-tenant-1-console             LoadBalancer   10.106.103.247   <pending>     9443:32095/TCP   2d3h
minio-tenant-1-hl                  ClusterIP      None             <none>        9000/TCP         2d3h
minio-tenant-1-log-hl-svc          ClusterIP      None             <none>        5432/TCP         2d3h
minio-tenant-1-log-search-api      ClusterIP      10.103.5.235     <none>        8080/TCP         2d3h
minio-tenant-1-prometheus-hl-svc   ClusterIP      None             <none>        9090/TCP         7h39m

• The minio service corresponds to the MinIO Tenant service. Applications should use this service for performing operations against the MinIO Tenant.

• The *-console service corresponds to the MinIO Console. Administrators should use this service for accessing the MinIO Console and performing administrative operations on the MinIO Tenant.

The remaining services support Tenant operations and are not intended for consumption by users or administrators.

By default each service is visible only within the Kubernetes cluster. Applications deployed inside the cluster can access the services using the CLUSTER-IP.

Applications external to the Kubernetes cluster can access the services using the EXTERNAL-IP. This value is only populated for Kubernetes clusters configured for Ingress or a similar network access service. Kubernetes provides multiple options for configuring external access to services. See the Kubernetes documentation on Publishing Services (ServiceTypes) and Ingress for more complete information on configuring external access to services.

You can temporarily expose each service using the kubectl port-forward utility. Run the following examples to forward traffic from the local host running kubectl to the services running inside the Kubernetes cluster.

kubectl port-forward service/minio 443:443

kubectl port-forward service/minio-tenant-1-console 9443:9443