Table of Contents
MinIO Object Lifecycle Management allows creating rules for time or date based automatic transition or expiry of objects. For object transition, MinIO automatically moves the object to a configured remote storage tier. For object expiry, MinIO automatically deletes the object.
MinIO lifecycle management is built for behavior and syntax compatibility with AWS S3 Lifecycle Management. For example, you can export S3 lifecycle management rules and import them into MinIO or vice-versa. MinIO uses JSON to describe lifecycle management rules, and conversion to or from XML may be required.
MinIO supports creating object transition lifecycle management rules, where MinIO can automatically move an object to a remote storage “tier”. MinIO supports any S3-compatible service as a remote tier in addition to the following public cloud storage services:
MinIO object transition supports use cases like moving aged data from MinIO clusters in private or public cloud infrastructure to low-cost public cloud storage solutions.
MinIO manages retrieving tiered objects on-the-fly without any additional application-side logic. MinIO retains the minimum object metadata required to support retrieving objects transitioned to a remote tier. MinIO therefore requires exclusive access to the data on the remote storage tier. Object retrieval assumes no external mutation, migration, or deletion of stored objects. Object transition is not a replacement for backup/recovery strategies such as Bucket Replication.
mc admin tier command to create a remote target for tiering
data to a supported Cloud Service Provider object storage You can then use the
mc ilm add command with one of the following commandline options to
create new transition rules on a bucket:
mc ilm add --transition-dateto transition objects after a specified calendar date.
mc ilm add --transition-daysto transition object after a specified number of calendar days.
To transition noncurrent object versions, specify the
when creating the transition rule.
MinIO lifecycle management supports expiring objects on a bucket. Object
“expiration” involves performing a
DELETE operation on the object. For
example, you can create a lifecycle management rule to expire any object
older than 365 days.
mc ilm add command with one of the following commandline
options to create new expiration rules on a bucket:
mc ilm add --expiry-dateto expire objects after a specified calendar date.
mc ilm add --expiry-daysto expire objects after a specified number of calendar days.
MinIO applies the expiration option to only the current object version by creating a
DeleteMarkeras is normal with versioned delete.
To expire noncurrent object versions, specify the
--noncurrentversion-expiration-daysoption when creating the expiration rule.
MinIO does not expire
DeleteMarkerseven if no other versions of that object exist.
To expire delete markers when there are no remaining versions for that object, specify the
--expired-object-delete-markeroption when creating the expiration rule.
MinIO uses a built-in scanner to actively check objects against all configured lifecycle management rules. The scanner is a low-priority process that yields to high IO workloads to prevent performance spikes triggered by rule timing. The scanner may therefore not detect an object as eligible for a configured transition or expiration lifecycle rule until after the lifecycle rule period has passed.
Delayed application of lifecycle management rules is typically associated to limited node resources and cluster size. Scanner speed tends to slow as clusters grow as more time is required to visit all buckets and objects. This can be exacerbated if the cluster hardware is undersized for regular workloads, as the scanner will yield to high cluster load to avoid performance loss. Consider regularly checking cluster metrics, capacity, and resource usage to ensure the cluster hardware is scaling alongside cluster and workload growth.