Deploy a MinIO Tenant

MinIO Kubernetes Operator and Plugin

The procedures on this page requires a valid installation of the MinIO Kubernetes Operator and assumes the local host has a matching installation of the MinIO Kubernetes Operator. This procedure assumes the latest stable Operator and Plugin version 5.0.14.

See Deploy the MinIO Operator for complete documentation on deploying the MinIO Operator.

You can install the MinIO Kubernetes plugin by downloading and installing the plugin binary to your local host:

curl https://github.com/minio/operator/releases/download/v5.0.14/kubectl-minio_5.0.14_linux_amd64 -o kubectl-minio
chmod +x kubectl-minio
mv kubectl-minio /usr/local/bin/

oc minio version

Invoke-WebRequest -Uri "https://github.com/minio/operator/releases/download/v5.0.14/kubectl-minio_5.0.14_windows_amd64.exe" -OutFile "C:\kubectl-plugins\kubectl-minio.exe"

oc minio version

OpenShift 4.7+ and oc CLI Tool

This procedure assumes installation of the MinIO Operator using the OpenShift 4.7+ and the OpenShift OperatorHub.

This procedure assumes your local machine has the OpenShift oc CLI tool installed and configured for access to the OpenShift Cluster. Download and Install the OpenShift CLI oc for use in this procedure.

See Deploy MinIO Operator on RedHat OpenShift for more complete instructions.

Check Security Context Constraints

The MinIO Operator deploys pods using the following default Security Context per pod:

securityContext:
  runAsUser: 1000
  runAsGroup: 1000
  runAsNonRoot: true
  fsGroup: 1000

Certain OpenShift Security Context Constraints limit the allowed UID or GID for a pod such that MinIO cannot deploy the Tenant successfully. Ensure that the Project in which the Operator deploys the Tenant has sufficient SCC settings that allow the default pod security context. You can alternatively modify the tenant security context settings during deployment.

The following command returns the optimal value for the securityContext:

oc get namespace <namespace> \
-o=jsonpath='{.metadata.annotations.openshift\.io/sa\.scc\.supplemental-groups}{"\n"}'

The command returns output similar to the following:

1056560000/10000

Take note of this value before the slash for use in this procedure.

Persistent Volumes

MinIO can use any Kubernetes Persistent Volume (PV) that supports the ReadWriteOnce access mode. MinIO’s consistency guarantees require the exclusive storage access that ReadWriteOnce provides. Additionally, MinIO recommends setting a reclaim policy of Retain for the PVC StorageClass. Where possible, configure the Storage Class, CSI, or other provisioner underlying the PV to format volumes as XFS to ensure best performance.

For Kubernetes clusters where nodes have Direct Attached Storage, MinIO strongly recommends using the DirectPV CSI driver. DirectPV provides a distributed persistent volume manager that can discover, format, mount, schedule, and monitor drives across Kubernetes nodes. DirectPV addresses the limitations of manually provisioning and monitoring local persistent volumes.

1) Access the MinIO Operator Console

Port Forwarding

Note

Some Kubernetes deployments may experience issues with timeouts during port-forwarding operations with the Operator Console. Select the NodePorts section to view instructions for alternative access. You can alternatively configure your preferred Ingress to grant access to the Operator Console service. See https://github.com/kubernetes/kubectl/issues/1368 for more information.

Run the kubectl minio proxy command to temporarily forward traffic from the MinIO Operator Console service to your local machine:

oc minio proxy

The command output includes a required token for logging into the Operator Console.

You can deploy a new MinIO Tenant from the Operator Dashboard.

kubectl get svc/console -n minio-operator -o json | jq -r '.spec.ports'

[
   {
      "name": "http",
      "nodePort": 31055,
      "port": 9090,
      "protocol": "TCP",
      "targetPort": 9090
   },
   {
      "name": "https",
      "nodePort": 31388,
      "port": 9443,
      "protocol": "TCP",
      "targetPort": 9443
   }
]

kubectl get secret/console-sa-secret -n minio-operator -o json | jq -r '.data.token' | base64 -d

Open your browser to the specified URL and enter the JWT Token into the login page. You should see the Tenants page:

Click the + Create Tenant to start creating a MinIO Tenant.

2) Complete the Tenant Setup

The Setup pane displays core configuration settings for the MinIO Tenant.

Settings marked with an asterisk * are required:

Select Create to create the Tenant using the current configuration. While all subsequent sections are optional, MinIO recommends reviewing them prior to deploying the Tenant.

3) The Configure Section

The Configure section displays optional configuration settings for the MinIO Tenant and its supporting services.

4) The Images Section

The Images section displays container image settings used by the MinIO Tenant.

5) The Pod Placement Section

The Pod Placement section displays pod scheduler settings for the MinIO Tenant.

6) The Identity Provider Section

The Identity Provider section displays the Identity Provider settings for the MinIO Tenant. This includes configuring an external IDP such as OpenID or Active Directory / LDAP.

7) The Security Section

The Security section displays TLS certificate settings for the MinIO Tenant.

8) The Encryption Section

The Encryption section displays the Server-Side Encryption (SSE) settings for the MinIO Tenant.

Enabling SSE also creates MinIO Key Encryption Service pods in the Tenant to facilitate SSE operations.

9) Audit Log Settings

10) Monitoring Settings

11) Deploy and View the Tenant

Select Create at any time to begin the deployment process. The MinIO Operator displays the root user credentials once as part of deploying the Tenant. Copy these credentials to a secure location.

You can monitor the Tenant creation process from the Tenants view. The State column updates throughout the deployment process.

Tenant deployment can take several minutes to complete. Once the State reads as Initialized, click the Tenant to view its details.

Each tab provides additional details or configuration options for the MinIO Tenant.

• METRICS - Displays metrics collected from the MinIO Tenant.

• SECURITY - Provides TLS-related configuration options.

• POOLS - Supports expanding the tenant by adding more Server Pools.

• LICENSE - Enter your SUBNET license.

12) Connect to the Tenant

The MinIO Operator creates services for the MinIO Tenant.

Use the oc get svc -n TENANT-PROJECT command to review the deployed services:

oc get svc -n minio-tenant-1

NAME                               TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
minio                              LoadBalancer   10.97.114.60     <pending>     443:30979/TCP    2d3h
minio-tenant-1-console             LoadBalancer   10.106.103.247   <pending>     9443:32095/TCP   2d3h
minio-tenant-1-hl                  ClusterIP      None             <none>        9000/TCP         2d3h
minio-tenant-1-log-hl-svc          ClusterIP      None             <none>        5432/TCP         2d3h
minio-tenant-1-log-search-api      ClusterIP      10.103.5.235     <none>        8080/TCP         2d3h
minio-tenant-1-prometheus-hl-svc   ClusterIP      None             <none>        9090/TCP         7h39m

• The minio service corresponds to the MinIO Tenant service. Applications should use this service for performing operations against the MinIO Tenant.

• The *-console service corresponds to the MinIO Console. Administrators should use this service for accessing the MinIO Console and performing administrative operations on the MinIO Tenant.

The remaining services support Tenant operations and are not intended for consumption by users or administrators.

By default each service is visible only within the Kubernetes cluster. Applications deployed inside the cluster can access the services using the CLUSTER-IP.

Applications external to the Kubernetes cluster can access the services using the EXTERNAL-IP. This value is only populated for Kubernetes clusters configured for Ingress or a similar network access service. Kubernetes provides multiple options for configuring external access to services.

See the OpenShift documentation on Route or Ingress for more complete information on configuring external access to services.

1) Access the MinIO Operator Interface

You can find the MinIO Operator Interface from the Operators left-hand navigation header.

1. Go to Operators, then Installed Operators.

2. For the Project dropdown, select openshift-operators.

3. Select MinIO Operators from the list of installed operators.

Click Create Tenant to begin the Tenant Creation process.

2) Create the Tenant

The Form View provides a user interface for configuring the new MinIO Tenant.

• Ensure the Tenant Secret -> Name is set to the name of the MinIO Root User Kubernetes Secret created as part of the prerequisites.

• Ensure the Console -> Console Secret -> Name is set to the name of the MinIO Console Kubernetes Secret created as part of the prerequisites.

You can also use the YAML view to perform more granular configuration of the MinIO Tenant. Refer to the MinIO Custom Resource Definition Documentation for guidance on setting specific fields. MinIO also publishes examples for additional guidance in creating custom Tenant YAML objects. Note that the OperatorHub YAML view supports creating only the MinIO Tenant object. Do not specify any other objects as part of the YAML input.

Changes to one view are reflected in the other. For example, you can make modifications in the YAML View and see those changes in the Form View.

Click Create to create the MinIO Tenant using the specified configuration. Use the credentials specified as part of the MinIO Root User secret to access the MinIO Server.

3) Connect to the Tenant

The MinIO Operator creates services for the MinIO Tenant. Use the oc get svc -n NAMESPACE command to review the deployed services:

oc get svc -n minio-tenant-1

NAME                               TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
minio                              LoadBalancer   10.97.114.60     <pending>     443:30979/TCP    2d3h
minio-tenant-1-console             LoadBalancer   10.106.103.247   <pending>     9443:32095/TCP   2d3h
minio-tenant-1-hl                  ClusterIP      None             <none>        9000/TCP         2d3h
minio-tenant-1-log-hl-svc          ClusterIP      None             <none>        5432/TCP         2d3h
minio-tenant-1-log-search-api      ClusterIP      10.103.5.235     <none>        8080/TCP         2d3h
minio-tenant-1-prometheus-hl-svc   ClusterIP      None             <none>        9090/TCP         7h39m

• The minio service corresponds to the MinIO Tenant service. Applications should use this service for performing operations against the MinIO Tenant.

• The *-console service corresponds to the MinIO Console. Administrators should use this service for accessing the MinIO Console and performing administrative operations on the MinIO Tenant.

The remaining services support Tenant operations and are not intended for consumption by users or administrators.

By default each service is visible only within the Kubernetes cluster. Applications deployed inside the cluster can access the services using the CLUSTER-IP.

Applications external to the Kubernetes cluster can access the services using the EXTERNAL-IP. This value is only populated for Kubernetes clusters configured for Ingress or a similar network access service. Kubernetes provides multiple options for configuring external access to services. See the Kubernetes documentation on Publishing Services (ServiceTypes) and Ingress for more complete information on configuring external access to services.