Welcome to the upcoming version of the MinIO Documentation! The content of these pages may change at any time. If you can't find what you're looking for, check our legacy documentation. Thank you for your patience.

Deploy MinIO Operator on Kubernetes

Table of Contents

Overview

MinIO is a Kubernetes-native high performance object store with an S3-compatible API. The MinIO Kubernetes Operator supports deploying MinIO Tenants onto private and public cloud infrastructures (“Hybrid” Cloud).

The following procedure installs the latest stable version (v4.1.1) of the MinIO Operator and MinIO Plugin on Kubernetes infrastructure:

  • The MinIO Operator installs a Custom Resource Document (CRD) to support describing MinIO tenants as a Kubernetes object. See the MinIO Operator CRD Reference for complete documentation on the MinIO CRD.

  • The MinIO Kubernetes Plugin brings native support for deploying and managing MinIO tenants on a Kubernetes cluster using the kubectl minio command.

This procedure assumes a generic Kubernetes environment. The following procedures provide more specific guidance for certain Kubernetes providers:

This documentation assumes familiarity with all referenced Kubernetes concepts, utilities, and procedures. While this documentation may provide guidance for configuring or deploying Kubernetes-related resources on a best-effort basis, it is not a replacement for the official Kubernetes Documentation.

Prerequisites

Kubernetes Version 1.19.0

Starting with v4.0.0, the MinIO Operator and MinIO Kubernetes Plugin require Kubernetes 1.19.0 and later. The Kubernetes infrastructure and the kubectl CLI tool must have the same version of 1.19.0+.

Prior to v4.0.0, the MinIO Operator and Plugin required Kubernetes 1.17.0. You must upgrade your Kubernetes infrastructure to 1.19.0 or later to use the MiNIO Operator or Plugin v4.0.0 or later.

Kubernetes krew

This procedure uses the Kubernetes krew plugin manager for installing the MinIO Kubernetes Operator and Plugin.

See the krew installation documentation for specific instructions on installation.

Kubernetes TLS Certificate API

The MinIO Operator automatically generates TLS Certificate Signing Requests (CSR) and uses the Kubernetes certificates.k8s.io TLS certificate management API to create signed TLS certificates.

The MinIO Operator therefore requires that the Kubernetes kube-controller-manager configuration include the following configuration settings:

  • --cluster-signing-key-file - Specify the PEM-encoded RSA or ECDSA private key used to sign cluster-scoped certificates.

  • --cluster-signing-cert-file - Specify the PEM-encoded x.509 Certificate Authority certificate used to issue cluster-scoped certificates.

The Operator cannot complete initialization if the Kubernetes cluster is not configured to respond to a generated CSR. Certain Kubernetes providers do not specify these configuration values by default.

To verify whether the kube-controller-manager has the required settings, use the following command. Replace $CLUSTER-NAME with the name of the Kubernetes cluster:

kubectl get pod kube-controller-manager-$CLUSTERNAME-control-plane \
  -n kube-system -o yaml

Confirm that the output contains the highlighted lines. The output of the example command above may differ from the output in your terminal:

 spec:
 containers:
 - command:
     - kube-controller-manager
     - --allocate-node-cidrs=true
     - --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
     - --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
     - --bind-address=127.0.0.1
     - --client-ca-file=/etc/kubernetes/pki/ca.crt
     - --cluster-cidr=10.244.0.0/16
     - --cluster-name=my-cluster-name
     - --cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt
     - --cluster-signing-key-file=/etc/kubernetes/pki/ca.key
 ...

Important

The MinIO Operator automatically generates TLS certificates for all MinIO Tenant pods using the specified Certificate Authority (CA). Clients external to the Kubernetes cluster must trust the Kubernetes cluster CA to connect to the MinIO Operator or MinIO Tenants.

Clients which cannot trust the Kubernetes cluster CA can try disabling TLS validation for connections to the MinIO Operator or a MinIO Tenant.

Alternatively, you can generate x.509 TLS certificates signed by a known and trusted CA and pass those certificates to MinIO Tenants. See User-Generated TLS Certificates for MinIO Object Storage for more complete documentation.

Procedure

1) Install the MinIO Kubernetes Operator

Run the following commands to install the MinIO Operator and Plugin using the Kubernetes krew plugin manager:

kubectl krew update
kubectl krew install minio

2) Initialize the MinIO Kubernetes Operator

Run the kubectl minio init command to initialize the MinIO Operator:

kubectl minio init

The command initializes the MinIO Operator with the following default settings:

Important

Document all arguments used when initializing the MinIO Operator.

3) Validate the Operator Installation

To verify the installation, run the following command:

kubectl get all --namespace minio-operator

If you initialized the Operator with a custom namespace, replace minio-operator with that namespace.

The output resembles the following:

NAME                                  READY   STATUS    RESTARTS   AGE
pod/console-59b769c486-cv7zv          1/1     Running   0          81m
pod/minio-operator-7976b4df5b-rsskl   1/1     Running   0          81m

NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/console    ClusterIP   10.105.218.94    <none>        9090/TCP,9443/TCP   81m
service/operator   ClusterIP   10.110.113.146   <none>        4222/TCP,4233/TCP   81m

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/console          1/1     1            1           81m
deployment.apps/minio-operator   1/1     1            1           81m

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/console-59b769c486          1         1         1       81m
replicaset.apps/minio-operator-7976b4df5b   1         1         1       81m