Welcome to the upcoming version of the MinIO Documentation! The content on this page is under active development and may change at any time. If you can't find what you're looking for, check our legacy documentation. Thank you for your patience.

mc mirror

Description

The mc mirror command synchronizes content to an S3-compatible host, similar to the rsync utility. mc mirror supports both filesystems and S3-compatible hosts as the synchronization source.

MinIO Trims Empty Prefixes on Object Removal

The mc mirror --watch command continuously synchronizes the source and destination targets. This includes automatically removing objects on the destination if they are removed on the source.

mc mirror --watch relies on the mc removal API for deleting objects. As part of removing the last object in a bucket prefix, mc also recursively removes each empty part of the prefix up to the bucket root. mc only applies the recursive removal to prefixes created implicitly as part of object write operations - that is, the prefix was not created using an explicit directory creation command such as mc mb.

For example, consider a bucket photos with the following object prefixes:

  • photos/2021/january/myphoto.jpg

  • photos/2021/february/myotherphoto.jpg

  • photos/NYE21/NewYears.jpg

photos/NYE21 is the only prefix explicitly created using mc mb. All other prefixes were implicitly created as part of writing the object located at that prefix.

If an mc command removes myphoto.jpg, the removal API automatically trims the empty /january prefix. If a subsequent mc command removes myotherphoto.jpg, the removal API automatically trims both the /february prefix and the now-empty /2021 prefix. If an mc command removes NewYears.jpg, the /NYE21 prefix remains in place since it was explicitly created.

If using mc mirror --watch for operations on a filesystem, mc applies this same behavior by recursively trimming empty directory paths up to the root. However, the mc remove API cannot distinguish between an explicitly created directory path and an implicitly created one. If mc mirror --watch deletes the last object at a filesystem path, mc recursively deletes all empty directories within that path up to the root as part of the removal operation.

Examples

Mirror a Local Directory to an S3-Compatible Host

Use mc mirror to mirror files from a filesystem to an S3 Host:

mc mirror FILEPATH ALIAS/PATH
  • Replace FILEPATH with the full file path to the directory to mirror.

  • Replace ALIAS with the alias of a configured S3-compatible host.

  • Replace PATH with the destination bucket.

Continuously Mirror a Local Directory to an S3-Compatible Host

Use mc watch with --watch to continuously mirror files from a filesystem to an S3-compatible host:

mc mirror FILEPATH ALIAS/PATH
  • Replace FILEPATH with the full file path to the directory to mirror.

  • Replace ALIAS with the alias of a configured S3-compatible host.

  • Replace PATH with the destination bucket.

Continuously Mirror S3 Bucket to an S3-Compatible Host

Use mc mirror with --watch to continuously mirror objects in a bucket on one S3-compatible host to another S3-compatible host.

mc mirror --watch SRCALIAS/SRCPATH TGTALIAS/TGTPATH
  • Replace SRCALIAS with alias of a configured S3-compatible host.

  • Replace SRCPATH with the bucket to mirror.

  • Replace TGTALIAS with the alias of a configured S3-compatible host.

  • Replace TGTPATH with the destination bucket.

Syntax

mirror has the following syntax:

mc mirror [FLAGS] SOURCE TARGET

mirror supports the following arguments:

SOURCE

REQUIRED

The file(s) or object(s) to synchronize to the TARGET S3 host.

For objects on S3-compatible hosts, specify the path to the object as ALIAS/PATH, where:

  • ALIAS is the alias of a configured S3-compatible host, and

  • PATH is the path to the bucket or object. If specifying a bucket, mc mirror synchronizes all objects in the bucket.

mc mirror [FLAGS] play/mybucket/ TARGET

For files on a filesystem, specify the full filesystem path to the file or directory :

mc mirror [FLAGS] ~/data/ TARGET

If specifying a directory, mc mirror synchronizes all files in the directory.

TARGET

REQUIRED

The full path to bucket in which mc mirror copies synchronized SOURCE objects. Specify the TARGET as ALIAS/PATH, where:

  • ALIAS is the alias of a configured S3-compatible host, and

  • PATH is the path to the bucket.

mc mirror SOURCE play/mybucket

mc mirror uses the object or file names from the SOURCE when synchronizing to the TARGET bucket.

--overwrite

Overwrites object(s) on the TARGET.

--remove

Removes extraneous object(s) on the target.

--watch, --w

Continuously monitor the SOURCE path and synchronize changes.

--region

Specify the string region when creating new bucket(s) on the target.

Defaults to "us-east-1".

--preserve, --a

Preserve file system attributes and bucket policy rules of the SOURCE on the TARGET.

--exclude

Exclude object(s) in the SOURCE path that match the specified object name pattern.

--older-than

Mirror object(s) older than the specified time limit. Specify a string in #d#hh#mm#ss format. For example: --older-than 1d2hh3mm4ss

Defaults to 0 (all objects).

--newer-than

Mirror object(s) newer than the specified number of days. Specify a string in #d#hh#mm#ss format. For example: --older-than 1d2hh3mm4ss

Defaults to 0 (all objects).

--fake

Perform a fake mirror operation. Use this operation to perform validate that the mc mirror operation will only mirror the desired objects or buckets.

--storage-class, --sc

Set the storage class for the new object(s) on the TARGET.

See the Amazon documentation on Storage Classes for more information on S3 storage classses.

--encrypt-key

Encrypt or decrypt objects using server-side encryption with client-specified keys. Specify key-value pairs as KEY=VALUE.

  • Each KEY represents a bucket or object.

  • Each VALUE represents the data key to use for encrypting

    object(s).

Enclose the entire list of key-value pairs passed to --encrypt-key in double quotes ".

--encrypt-key can use the MC_ENCRYPT_KEY environment variable for retrieving a list of encryption key-value pairs as an alternative to specifying them on the command line.

You can only delete encrypted objects if you specify the correct --encrypt-key secret key.