Deploy MinIO on Docker Swarm Slack Docker Pulls

Docker Engine provides cluster management and orchestration features in Swarm mode. MinIO server can be easily deployed in distributed mode on Swarm to create a multi-tenant, highly-available and scalable object store.

As of Docker Engine v1.13.0 (Docker Compose v3.0), Docker Swarm and Compose are cross-compatible. This allows a Compose file to be used as a template to deploy services on Swarm. We have used a Docker Compose file to create distributed MinIO setup.

1. Prerequisites

2. Create a Swarm

Create a swarm on the manager node by running

docker swarm init --advertise-addr <MANAGER-IP>

Once the swarm is initialized, you'll see the below response.

docker swarm join \
  --token  SWMTKN-1-49nj1cmql0jkz5s954yi3oex3nedyz0fb0xx14ie39trti4wxv-8vxv8rssmk743ojnwacrr2e7c \

You can now add worker nodes to the swarm by running the above command. Find detailed steps to create the swarm on Docker documentation site.

3. Create Docker secrets for MinIO

echo "AKIAIOSFODNN7EXAMPLE" | docker secret create access_key -
echo "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" | docker secret create secret_key -

4. Deploy distributed MinIO services

The example MinIO stack uses 4 Docker volumes, which are created automatically by deploying the stack. We have to make sure that the services in the stack are always (re)started on the same node, where the service is deployed the first time.
Otherwise Docker will create a new volume upon restart of the service on another Docker node, which is not in sync with the other volumes and the stack will fail to start healthy.
Before deploying the stack, add labels to the Docker nodes where you want the minio services to run:

docker node update --label-add minio1=true <DOCKER-NODE1>
docker node update --label-add minio2=true <DOCKER-NODE2>
docker node update --label-add minio3=true <DOCKER-NODE3>
docker node update --label-add minio4=true <DOCKER-NODE4>

It is possible to run more than one minio service on one Docker Node. Set the labels accordingly.

Download the Docker Compose file on your Swarm master. Then execute the command

docker stack deploy --compose-file=docker-compose-secrets.yaml minio_stack

This deploys services described in the Compose file as Docker stack minio_stack. Look up the docker stack command reference for more info.

After the stack is successfully deployed, you should be able to access MinIO server via MinIO Client mc or your browser at http://[Node_Public_IP_Address]:[Expose_Port_on_Host]

4. Remove distributed MinIO services

Remove the distributed MinIO services and related network by

docker stack rm minio_stack

Swarm doesn't automatically remove host volumes created for services. This may lead to corruption when a new MinIO service is created in the swarm. So, we recommend removing all the volumes used by MinIO, manually. To do this, logon to each node in the swarm and run

docker volume prune

This will remove all the volumes not associated with any container.

5. Accessing MinIO services

The services are exposed, by default, on the internal overlay network by their services names (minio1, minio2, ...).
The docker-compose.yml file also exposes the MinIO services behind a single alias on the minio_distributed network.

Services in the Swarm which are attached to that network can interact with the host "minio-cluster" instead of individual services' hostnames. This provides a simple way to loosely load balance across all the MinIO services in the Swarm as well as simplifies configuration and management.


Explore Further