Table of Contents
Site replication configures multiple independent MinIO deployments as a cluster of replicas called peer sites.
Site replication assumes the use of either the included MinIO identity provider (IDP) or an external IDP. All configured deployments must use the same IDP. Deployments using an external IDP must use the same configuration across sites.
Each MinIO deployment (“peer site”) synchronizes the following changes across the other peer sites:
Creation, modification, and deletion of buckets and objects, including
Bucket and Object Configurations
Locks, including retention and legal hold configurations
Creation and deletion of IAM users, groups, policies, and policy mappings to users or groups (for LDAP users or groups)
Creation of Security Token Service (STS) credentials for session tokens verifiable from the local root credentials
Creation and deletion of service accounts (except those owned by the root user)
Site replication enables bucket versioning for all new and existing buckets on all replicated sites.
MinIO deployments in a site replication configuration do not replicate the creation or modification of the following items:
After enabling site replication, identity and access management (IAM) settings sync in the following order:
Policies
User accounts (for local users)
Groups
Service accounts
Service accounts for root do not sync.
Policy mapping for synced user accounts
Policy mapping for Security Token Service (STS) users
Policies
Service accounts associated to OIDC accounts with a valid MinIO Policy. root service accounts do not sync.
Policy mapping for synced user accounts
Policy mapping for Security Token Service (STS) users
Policies
Groups
Service accounts associated to LDAP accounts with a valid MinIO Policy. root service accounts do not sync.
Policy mapping for synced user accounts
Policy mapping for Security Token Service (STS) users
After the initial synchronization of data across peer sites, MinIO continually replicates and synchronizes replicable data among all sites as they occur on any site.
Any MinIO deployment in the site replication configuration can resynchronize damaged replica-eligible data from the peer with the most updated (“latest”) version of that data.
Only one site can have data at the time of setup. The other sites must be empty of buckets and objects.
After configuring site replication, any data on the first deployment replicates to the other sites.
All sites must use the same Identity Provider. Site replication supports the included MinIO IDP, OIDC, or LDAP.
For SSE-S3 or SSE-KMS encryption via Key Management Service (KMS), all sites must have access to a central KMS deployment.
You can achieve this with a central KES server or multiple KES servers (say one per site) connected via a central supported key vault server.
Deploy two or more separate MinIO sites, using the same Identity Provider for each site
Only one site can have any buckets or objects on it. The other site(s) must be empty.
In a browser, access the Console for the site with data (if any)
For example, https://<addressforsite>:9000
Replace <addressforsite> with the IP address or URL for the MinIO deployment.
Select Settings, then Site Replication
Select Add Sites +
Complete the requested information for the site:
(required) The user name for root to use for signing in to the site.
(required) The password for root to use for signing in to the site.
A name or other identifying text to associate to the site.
(required) The URL or IP address and port to use to access the site.
To add additional sites beyond two, select the + button to the side of one of the Site entries.
To remove a site previously added, select the - button to the side of the site.
Site replication adds a svcacct under the root user to perform replication activities.
Select Save
Select Replication Status button to verify replication has completed across peer sites.
Any replicable data that exists should show as successfully synced.
For more on reviewing site replication, see the Site Replication Status tutorial.
The following steps create a new site replication configuration for three distributed deployments. One of the sites contains replicable data.
The three sites use aliases, minio1, minio2, and minio3, and only minio1 contains any data.
Deploy three or more separate MinIO sites, using the same IDP
Start with empty sites or have no more than one site with any replicable data.
Configure an alias for each site
For example, for three MinIO sites, you might create aliases minio1, minio2, and minio3.
Use mc alias set
Hybrid Cloud Object Storage
Baremetal
Erasure Code Calculator
Reference Hardware
mc alias set minio1 https://minio1.example.com:9000 adminuser adminpassword
mc alias set minio2 https://minio2.example.com:9000 adminuser adminpassword
mc alias set minio3 https://minio3.example.com:9000 adminuser adminpassword
or define environment variables
export MC_HOST_minio1=https://adminuser:adminpassword@minio1.example.com
export MC_HOST_minio2=https://adminuser:adminpassword@minio2.example.com
export MC_HOST_minio3=https://adminuser:adminpassword@minio3.example.com
Add site replication configuration
mc admin replicate add minio1 minio2 minio3
If all sites are empty, the order of the aliases does not matter. If one of the sites contains any replicable data, you must list it first.
No more than one site can contain any replicable data.
Query the site replication configuration to verify
mc admin replicate info minio1
You can use the alias for any peer site in the site replication configuration.
Query the site replication status to confirm any initial data has replicated to all peer sites.
mc admin replicate status minio1
You can use the alias for any of the peer sites in the site replication configuration. The output should say that all replicable data is in sync.
The output could resemble the following:
Bucket replication status:
● 1/1 Buckets in sync
Policy replication status:
● 5/5 Policies in sync
User replication status:
No Users present
Group replication status:
No Groups present
For more on reviewing site replication, see the Site Replication Status tutorial.
You can add more sites to an existing site replication configuration.
The new site must meet the following requirements:
Site is fully deployed and accessible by hostname or IP
Shares the IDP configuration as all other sites in the configuration
Uses the same root user credentials as other configured sites
Contains no bucket or object data
Deploy a new, empty MinIO site
In a browser, access the Console for one of the exisitng replicated sites
For example, https://<addressforsite>:9000
Select Settings, then Site Replication
Select Add Sites +
Make the following entries:
(required) The user name to use for signing in to each site. Should be the same across all sites.
(required) The password for the user name to use for signing in to each site. Should be the same across all sites.
An alias to use for the site name.
(required) The URL or IP address and port to use to access the site.
To add additional sites beyond two, select the + button to the side of the last Site entry.
Select Save
Deploy three or more separate MinIO sites, using the same external IDP
Only one site can have any buckets or objects on it. The other sites must be empty.
Configure an alias for each site
To check the existing aliases, use mc alias list.
For example, for three MinIO sites, you might create aliases minio1, minio2, and minio3.
Use mc alias set
mc alias set minio1 https://minio1.example.com:9000 adminuser adminpassword
mc alias set minio2 https://minio2.example.com:9000 adminuser adminpassword
mc alias set minio3 https://minio3.example.com:9000 adminuser adminpassword
or define environment variables
export MC_HOST_minio1=https://adminuser:adminpassword@minio1.example.com
export MC_HOST_minio2=https://adminuser:adminpassword@minio2.example.com
export MC_HOST_minio3=https://adminuser:adminpassword@minio3.example.com
Add site replication configuration
List all existing replicated sites first, then list the new site(s) to add.
In this example, minio1, minio2, and minio3 are already configured for replication.
The command adds minio4 and minio5 as new sites to add to the replication.
minio4 and minio5 must be empty.
mc admin replicate add minio1 minio2 minio3 minio4 minio5
Query the site replication configuration to verify
mc admin replicate info minio1
If a peer site changes its hostname, you can modify the replication configuration to reflect the new hostname.
In a browser, access the Console for one of the replicated sites
For example, https://<addressforsite>:9000
Select Settings, then Site Replication
Select the pencil Edit icon to the side of the site to update
Make the following entries:
(required) The new endpoint address and port to use.
Select Update
Obtain the site’s Deployment ID with mc admin replicate info
mc admin replicate info <ALIAS>
Update the site’s endpoint with mc admin replicate edit
mc admin replicate edit ALIAS --deployment-id [DEPLOYMENT-ID] --endpoint [NEW-ENDPOINT]
Replace [DEPLOYMENT-ID] with the deployment ID of the site to update.
Replace [NEW-ENDPOINT] with the new endpoint for the site.
You can remove a site from replication at any time. You can re-add the site at a later date, but you must first completely wipe bucket and object data from the site.
In a browser, access the Console for one of the replicated sites
For example, https://<addressforsite>:9000
Select Settings, then Site Replication
Select the trash can Delete icon to the side of the site to update
Confirm the site deletion at the prompt by selecting Delete
mc admin replicate remove <ALIAS> --all --force
The -all flag removes the site as a peer from all participating sites.
The --force flag is required to removes the site from the site replication configuration.
MinIO provides information on replication across the sites for users, groups, policies, or buckets.
The summary information includes the number of Synced and Failed items for each category.
In a browser, access the Console for one of the replicated sites
For example, https://<addressforsite>:9000
Select Settings, then Site Replication
Select Replication Status
(Optional) View the replication status for a specific item
Select the type of item to view in the View Replication Status for a: dropdown
Specify the name of the specific Bucket, Group, Policy, or User to view
(Optional) Update the information by selecting Refresh
mc admin replicate status <ALIAS> --<flag> <value>
For example:
mc admin replicate status minio3 --bucket images
Displays the replication status for the images bucket on the minio3 site.
The output resembles the following:
● Bucket config replication summary for: images
Bucket | MINIO2 | MINIO3 | MINIO4
Tags | | |
Policy | | |
Quota | | |
Retention | | |
Encryption | | |
Replication | ✔ | ✔ | ✔
mc admin replicate status minio3 --all
Displays the replication status summary for all replication sites of which minio3 is part.
The output resembles the following:
Bucket replication status:
● 1/1 Buckets in sync
Policy replication status:
● 5/5 Policies in sync
User replication status:
● 1/1 Users in sync
Group replication status:
● 0/2 Groups in sync
Group | MINIO2 | MINIO3 | MINIO4
ittechs | ✗ in-sync | | ✗ in-sync
managers | ✗ in-sync | | ✗ in-sync
This work is licensed under a Creative Commons Attribution 4.0 International License.
©2020-Present, MinIO, Inc.
