Deploy Distributed MinIO AIStor on Docker Compose

Docker Compose lets you define and run a multi-container application on a single host. You can use a Compose file to start several MinIO AIStor server instances in separate containers on the same machine, forming a distributed deployment behind an NGINX load balancer.

• Familiarity with Docker Compose.
• Docker installed on your machine.
• A MinIO AIStor license file. MinIO AIStor requires a license to run. To request a free tier license, go to the MinIO AIStor pricing page and select Get Started under the Free tier. Save the license to minio.license in the same directory as the Compose file. For more information, see Licenses.

1. Create the Compose file

   The following docker-compose.yaml starts four MinIO AIStor server instances, each with two named data volumes, and an NGINX service that reverse proxies and load balances across them. The shared command starts each server with the http://minio{1...4}/data{1...2} server pool definition.

   Create docker-compose.yaml:

   # Settings and configurations that are common to all MinIO containers.
   # NOTE: Update the image tag to the latest MinIO AIStor release from
   # quay.io/minio/aistor/minio
   x-minio-common: &minio-common
     image: quay.io/minio/aistor/minio:latest
     command: server --console-address ":9001" http://minio{1...4}/data{1...2} --license /minio.license
     expose:
       - "9000"
       - "9001"
     # environment:
     #   MINIO_ROOT_USER: minioadmin
     #   MINIO_ROOT_PASSWORD: minioadmin
     healthcheck:
       test: ["CMD", "mc", "ready", "local"]
       interval: 5s
       timeout: 5s
       retries: 5
   
   # Starts four containers running MinIO AIStor server instances.
   # NGINX reverse proxies and load balances across them on ports 9000 and 9001.
   services:
     minio1:
       <<: *minio-common
       hostname: minio1
       volumes:
         - data1-1:/data1
         - data1-2:/data2
         - ./minio.license:/minio.license:ro
   
     minio2:
       <<: *minio-common
       hostname: minio2
       volumes:
         - data2-1:/data1
         - data2-2:/data2
         - ./minio.license:/minio.license:ro
   
     minio3:
       <<: *minio-common
       hostname: minio3
       volumes:
         - data3-1:/data1
         - data3-2:/data2
         - ./minio.license:/minio.license:ro
   
     minio4:
       <<: *minio-common
       hostname: minio4
       volumes:
         - data4-1:/data1
         - data4-2:/data2
         - ./minio.license:/minio.license:ro
   
     nginx:
       image: nginx:1.27-alpine
       hostname: nginx
       volumes:
         - ./nginx.conf:/etc/nginx/nginx.conf:ro
       ports:
         - "9000:9000"
         - "9001:9001"
       depends_on:
         - minio1
         - minio2
         - minio3
         - minio4
   
   # By default this configuration uses the local volume driver.
   # For custom volumes, replace with your volume driver configuration.
   volumes:
     data1-1:
     data1-2:
     data2-1:
     data2-2:
     data3-1:
     data3-2:
     data4-1:
     data4-2:
   

   The command uses server, not minio server, because the image entrypoint is minio. Each service mounts the same minio.license file so every server instance can validate the license.

2. Create the NGINX configuration

   Create nginx.conf in the same directory as the Compose file. It defines an upstream for the S3 API (port 9000) and an upstream for the Web Console (port 9001), then proxies to each.

   user  nginx;
   worker_processes  auto;
   
   error_log  /var/log/nginx/error.log warn;
   pid        /var/run/nginx.pid;
   
   events {
       worker_connections  4096;
   }
   
   http {
       include       /etc/nginx/mime.types;
       default_type  application/octet-stream;
   
       log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent "$http_referer" '
                         '"$http_user_agent" "$http_x_forwarded_for"';
   
       access_log  /var/log/nginx/access.log  main;
       sendfile        on;
       keepalive_timeout  65;
   
       upstream minio {
           server minio1:9000;
           server minio2:9000;
           server minio3:9000;
           server minio4:9000;
       }
   
       upstream console {
           ip_hash;
           server minio1:9001;
           server minio2:9001;
           server minio3:9001;
           server minio4:9001;
       }
   
       server {
           listen       9000;
           listen  [::]:9000;
           server_name  localhost;
   
           # To allow special characters in headers
           ignore_invalid_headers off;
           # Allow any size file to be uploaded.
           # Set to a value such as 1000m; to restrict file size to a specific value
           client_max_body_size 0;
           # To disable buffering
           proxy_buffering off;
           proxy_request_buffering off;
   
           location / {
               proxy_set_header Host $http_host;
               proxy_set_header X-Real-IP $remote_addr;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_set_header X-Forwarded-Proto $scheme;
   
               proxy_connect_timeout 300;
               # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
               proxy_http_version 1.1;
               proxy_set_header Connection "";
               chunked_transfer_encoding off;
   
               proxy_pass http://minio;
           }
       }
   
       server {
           listen       9001;
           listen  [::]:9001;
           server_name  localhost;
   
           # To allow special characters in headers
           ignore_invalid_headers off;
           # Allow any size file to be uploaded.
           # Set to a value such as 1000m; to restrict file size to a specific value
           client_max_body_size 0;
           # To disable buffering
           proxy_buffering off;
           proxy_request_buffering off;
   
           location / {
               proxy_set_header Host $http_host;
               proxy_set_header X-Real-IP $remote_addr;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_set_header X-Forwarded-Proto $scheme;
               proxy_set_header X-NginX-Proxy true;
   
               # This is necessary to pass the correct IP to be hashed
               real_ip_header X-Real-IP;
   
               proxy_connect_timeout 300;
   
               # To support websocket
               proxy_http_version 1.1;
               proxy_set_header Upgrade $http_upgrade;
               proxy_set_header Connection "upgrade";
   
               chunked_transfer_encoding off;
   
               proxy_pass http://console;
           }
       }
   }
   

3. Start the deployment

   From the directory containing docker-compose.yaml, nginx.conf, and minio.license, pull the images and start the containers. Docker Compose pulls the MinIO AIStor image, so there is no need to download a binary.

   docker compose pull
   docker compose up
   

   Alternatively, deploy the stack with Docker Swarm:

   docker stack deploy --compose-file docker-compose.yaml minio
   

4. Access the deployment

   The distributed instances are reachable on the host through NGINX:

   • S3 API: port 9000
   • Web Console: http://127.0.0.1:9001/

The example deploys four instances by default. You can add more MinIO AIStor services, up to a total of 16. To add a service:

• Copy an existing service definition and change the service name and hostname (for example, minio5).
• Update the command server pool definition in the shared x-minio-common anchor to match the new instance count (for example, http://minio{1...8}/data{1...2}).
• Add the new instance to the minio and console upstream blocks in nginx.conf.
• Add the new instance to the NGINX service depends_on list.