`mc replicate add`

Table of Contents

Syntax
- Parameters
- Global Flags
Examples
- Configure Bucket Replication
- Configure Bucket Replication for Historical Data Record
Behavior

Syntax

The mc replicate add command creates a new server-side replication rule for a bucket on a MinIO deployment.

The MinIO deployment automatically begins synchronizing new objects to the remote MinIO deployment after creating the rule. You can optionally configure synchronization of existing objects, delete operations, and fully-deleted objects.

This command requires first configuring the remote bucket target using the mc admin bucket remote add command. You must specify the resulting remote ARN as part of running mc replicate add.

EXAMPLE

The following command adds a new replication rule for the mydata bucket on the myminio MinIO deployment:

mc replicate add                                                                        \
   --remote-bucket "arn:minio:replication:aefc8b3a-1f6c-4d7a-86dc-1b0bdffa9100:mydata"  \
   --replicate "delete,delete-marker,existing-objects"                                  \
   myminio/mydata

The replication rule synchronizes versioned delete operations, delete markers, and existing objects to the remote MinIO deployment.

SYNTAX

The command has the following syntax:

mc [GLOBALFLAGS] replicate add               \
                 --remote-bucket "string"    \
                 [--disable]                 \
                 [--id "string"]             \
                 [--replicate "string"]      \
                 [--storage-class "string"]  \
                 [--tags "string"]           \
                 [--priority int]            \
                 ALIAS

Brackets [] indicate optional parameters.
Parameters sharing a line are mutually dependent.
Parameters seperated using the pipe | operator are mutually exclusive.

Copy the example to a text editor and modify as-needed before running the command in the terminal/shell.

Parameters

ALIAS

Required the alias of the MinIO deployment and full path to the bucket or bucket prefix on which to create the replication rule. For example:

mc replicate add --remote-bucket "arn:minio:replica::UUID" play/mybucket

--remote-bucket

Required Specify the ARN for the destination deployment and bucket. You can retrieve the ARN using mc admin bucket remote:

Use the mc admin bucket remote ls to retrieve a list of ARNs for the bucket on the destination deployment.
Use the mc admin bucket remote add to create a replication ARN for the bucket on the destination deployment.

--disable

Optional Creates the replication rule in the “disabled” state. MinIO does not begin replicating objects using the rule until it is enabled using mc replicate edit.

Objects created while replication is disabled are not immediately eligible for replication after enabling the rule. You must explicitly enable replication of existing objects by including "existing-objects" to the list of replication features specified to mc replicate edit --replicate. See Replication of Existing Objects for more information.

--id: Optional Specify a unique ID for the replication rule. MinIO automatically generates an ID if one is not specified.

--replicate

Optional Specify a comma-separated list of the following values to enable extended replication features.

delete - Directs MinIO to replicate DELETE operations to the destination bucket.
delete-marker - Directs MinIO to replicate delete markers to the destination bucket.
existing-objects - Directs MinIO to replicate objects created before replication was enabled or while replication was suspended.

--storage-class

Optional

Specify the MinIO storage class to apply to replicated objects.

--tags

Optional Specify one or more ampersand & separated key-value pair tags which MinIO uses for filtering objects to replicate. For example:

mc replicate add --tags "TAG1=VALUE&TAG2=VALUE&TAG3=VALUE" ALIAS

MinIO applies the replication rule to any object whose tag set contains the specified replication tags.

--priority

Optional Specify the integer priority of the replication rule. The value must be unique among all other rules on the source bucket. Higher values imply a higher priority than all other rules.

The default value is 0.

Global Flags

--debug

Optional

Enables verbose output to the console.

For example:

mc --debug COMMAND

--config-dir: Optional
The path to a JSON formatted configuration file that mc uses for storing data. See Configuration File for more information on how mc uses the configuration file.

--JSON

Optional

Enables JSON lines formatted output to the console.

For example:

mc --JSON COMMAND

--no-color: Optional
Disables the built-in color theme for console output. Useful for dumb terminals.

--quiet: Optional
Suppresses console output.

--insecure: Optional
Disables TLS/SSL certificate verification. Allows TLS connectivity to servers with invalid certificates. Exercise caution when using this option against untrusted S3 hosts.

--version: Optional
Displays the current version of mc.

--help: Optional
Displays a summary of command usage on the terminal.

Examples

Configure Bucket Replication

The following mc replicate add command creates a replication configuration that synchronizes all new objects, existing objects, delete operations, and delete markers to the remote target:

mc replicate add myminio/mybucket \
   --remote-bucket "arn:minio:replica::UUID" \
   --replicate "delete,delete-marker,existing-objects"

Replace myminio/mybucket with the ALIAS and full bucket path for which to create the replication configuration.
Replace the --remote-bucket value with the ARN of the remote target. Use mc admin bucket remote ls to list all configured remote replication targets.
The --replicate flag directs MinIO to replicate all delete operations, delete markers, and existing objects to the remote. See Replication of Delete Operations and Replication of Existing Objects for more information on replication behavior.

Configure Bucket Replication for Historical Data Record

The following mc replicate add command creates a new bucket replication configuration that synchronizes all new and existing objects to the remote target:

mc replicate add myminio/mybucket \
   --remote-bucket "arn:minio:replica::UUID" \
   --replicate "existing-objects"

Replace myminio/mybucket with the ALIAS and full bucket path for which to create the replication configuration.
Replace the --remote-bucket value with the ARN of the remote target. Use mc admin bucket remote ls to list all configured remote replication targets.
The --replicate flag directs MinIO to replicate all existing objects to the remote. See Replication of Existing Objects for more information on replication behavior.

The resulting remote copy represents a historical record of objects on the remote, where delete operations on the source have no effect on the remote copy.

Behavior

Server-Side Replication Requires MinIO Source and Destination

MinIO server-side replication only works between MinIO deployments. Both the source and destination deployments must run MinIO.

To configure replication between arbitrary S3-compatible services, use mc mirror.

Enable Versioning on Source and Destination Buckets

MinIO relies on the immutability protections provided by versioning to synchronize objects between the source and replication target.

Use the mc version suspend command to enable versioning on both the source and destination bucket before starting this procedure:

mc version ALIAS/PATH

Replace ALIAS with the alias of the MinIO deployment.
Replace PATH with the bucket on which to enable versioning.

Required Permissions

MinIO strongly recommends creating users specifically for supporting bucket replication operations. See mc admin user and mc admin policy for more complete documentation on adding users and policies to a MinIO deployment.

Replication Admin

The following policy provides permissions for configuring and enabling replication on a deployment.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "admin:SetBucketTarget",
                "admin:GetBucketTarget"
            ],
            "Effect": "Allow",
            "Sid": "EnableRemoteBucketConfiguration"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:GetBucketLocation",
                "s3:GetBucketVersioning",
                "s3:GetObjectRetention",
                "s3:GetObjectLegalHold",
                "s3:PutReplicationConfiguration"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ],
            "Sid": "EnableReplicationRuleConfiguration"
        }
    ]
}

The "EnableRemoteBucketConfiguration" statement grants permission for creating a remote target for supporting replication.
The "EnableReplicationRuleConfiguration" statement grants permission for creating replication rules on a bucket. The "arn:aws:s3:::* resource applies the replication permissions to any bucket on the source deployment. You can restrict the user policy to specific buckets as-needed.

Use the mc admin policy add to add this policy to each deployment acting as a replication source. Use mc admin user add to create a user on the deployment and mc admin policy set to associate the policy to that new user.

Replication Remote User

The following policy provides permissions for enabling synchronization of replicated data into the deployment.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:GetBucketLocation",
                "s3:GetBucketVersioning",
                "s3:GetBucketObjectLockConfiguration",
                "s3:GetEncryptionConfiguration"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ],
            "Sid": "EnableReplicationOnBucket"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ReplicateTags",
                "s3:AbortMultipartUpload",
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionTagging",
                "s3:PutObject",
                "s3:PutObjectRetention",
                "s3:PutBucketObjectLockConfiguration",
                "s3:PutObjectLegalHold",
                "s3:DeleteObject",
                "s3:ReplicateObject",
                "s3:ReplicateDelete"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ],
            "Sid": "EnableReplicatingDataIntoBucket"
        }
    ]
}

The "EnableReplicationOnBucket" statement grants permission for a remote target to retrieve bucket-level configuration for supporting replication operations on all buckets in the MinIO deployment. To restrict the policy to specific buckets, specify those buckets as an element in the Resource array similar to "arn:aws:s3:::bucketName".
The "EnableReplicatingDataIntoBucket" statement grants permission for a remote target to synchronize data into any bucket in the MinIO deployment. To restrict the policy to specific buckets, specify those buckets as an element in the Resource array similar to "arn:aws:s3:::bucketName/*".

Use the mc admin policy add to add this policy to each deployment acting as a replication target. Use mc admin user add to create a user on the deployment and mc admin policy set to associate the policy to that new user.

Replication of Existing Objects

Starting with mc RELEASE.2021-06-13T17-48-22Z and minio RELEASE.2021-06-07T21-40-51Z,

MinIO supports automatically replicating existing objects in a bucket. MinIO existing object replication implements functionality similar to AWS Replicating existing objects between S3 buckets without the overhead of contacting technical support.

To enable replication of existing objects when creating a new replication rule, include "existing-objects" to the list of replication features specified to mc replicate add --replicate.
To enable replication of existing objects for an existing replication rule, add "existing-objects" to the list of existing replication features using mc replicate add --replicate. You must specify all desired replication features when editing the replication rule.

See Replication of Existing Objects for more complete documentation on this behavior.

Synchronization of Metadata Changes

MinIO supports two-way active-active replication configurations, where MinIO synchronizes new and modified objects between a bucket on two MinIO deployments. Starting with mc

RELEASE.2021-05-18T03-39-44Z, MinIO by default synchronizes metadata-only changes to a replicated object back to the “source” deployment. Prior to the this update, MinIO did not support synchronizing metadata-only changes to a replicated object.

With metadata synchronization enabled, MinIO resets the object replication status to indicate replication eligibility. Specifically, when an application performs a metadata-only update to an object with the REPLICA status, MinIO marks the object as PENDING and eligible for replication.

To disable metadata synchronization, use the mc replicate edit --replicate command and omit replica-metadata-sync from the replication feature list.

Replication of Delete Operations

MinIO supports replicating delete operations onto the target bucket. Specifically, MinIO can replicate both Delete Markers and the deletion of specific versioned objects:

For delete operations on an object, MinIO replication also creates the delete marker on the target bucket.
For delete operations on versions of an object, MinIO replication also deletes those versions on the target bucket.

MinIO does not replicate objects deleted due to lifecycle management expiration rules. MinIO only replicates explicit client-driven delete operations.

MinIO requires explicitly enabling replication of delete operations using the mc replicate add --replicate flag. This procedure includes the required flags for enabling replication of delete operations and delete markers. See Replication of Delete Operations for more complete documentation on this behavior.

Replication of Encrypted Objects

MinIO supports replicating objects encrypted with automatic Server-Side Encryption (SSE-S3). Both the source and destination buckets must have automatic SSE-S3 enabled for MinIO to replicate an encrypted object.

As part of the replication process, MinIO decrypts the object on the source bucket and transmits the unencrypted object. The destination MinIO deployment then re-encrypts the object using the destination bucket SSE-S3 configuration. MinIO strongly recommends enabling TLS on both source and destination deployments to ensure the safety of objects during transmission.

MinIO does not support replicating client-side encrypted objects (SSE-C).

S3 Compatibility

The mc commandline tool is built for compatibility with the AWS S3 API and is tested MinIO and AWS S3 for expected functionality and behavior.

MinIO provides no guarantees for other S3-compatible services, as their S3 API implementation is unknown and therefore unsupported. While mc commands may work as documented, any such usage is at your own risk.