Heal Settings

This page covers settings that control data healing behavior for the MinIO AIStor process. Healing automatically detects and repairs corrupted or missing data using erasure coding.

You can establish or modify settings by defining:

  • an environment variable on the host system prior to starting or restarting the AIStor Server. Refer to your operating system’s documentation for how to define an environment variable.
  • a configuration setting using mc admin config set.

If you define both an environment variable and the similar configuration setting, MinIO AIStor uses the environment variable value.

Some settings have only an environment variable or a configuration setting, but not both.

Each configuration setting controls fundamental MinIO AIStor behavior and functionality. Test configuration changes in a lower environment, such as DEV or QA, before applying to production.

Bitrot scan

Enable bitrot scanning on local drives when the scanner checks objects. Bitrot scanning verifies data integrity by comparing stored checksums against computed values.

Valid values:

  • off - Disable bitrot scanning.
  • on - Enable continuous bitrot scanning every scanner cycle.
  • A duration such as 1m, 6m, or 1y - Enable bitrot scanning at the specified interval. Minimum interval is 1m (one month). Use d for days, m for months, y for years.

The default value is off.

Maximum sleep

Set the maximum sleep duration between objects to slow down heal operations. Use this to reduce the I/O impact of healing on production workloads.

Specify a Go-compatible duration string such as 50ms, 100ms, or 1s.

The default value is 50ms.

Maximum I/O

Set the maximum number of I/O requests allowed between objects during healing. Together with maximum sleep, this controls healing throughput.

The default value is 50.

Dangling object handling

Controls how MinIO AIStor handles dangling (stale) objects that no longer have valid erasure set references.

Valid values:

  • purge - Immediately remove dangling objects.
  • quarantine - Move dangling objects to a quarantine location. Objects remain quarantined for the duration specified by dangling expiry.
  • ignore - Take no action on dangling objects.

The default value is purge.

Transitioned (tiered) objects

Objects transitioned to a remote (warm) tier through lifecycle tiering are never treated as dangling and are never purged, regardless of how many xl.meta copies are lost on the primary (hot) tier drives.

For these objects, the data resides in the warm tier while only the xl.meta metadata is stored on the primary tier. Healing therefore restores only the xl.meta metadata. Before healing, AIStor verifies that the warm-tier content is accessible by performing a small sample read. As long as at least one xl.meta copy and the tier configuration remain available, AIStor rebuilds the metadata to all drives. If the warm tier is inaccessible, healing fails and requires manual intervention.

The minio_debug_heal_transitioned_object_total metric reports the number of transitioned objects processed by healing.

Dangling expiry

Set the expiration duration for quarantined dangling objects. Only applies when dangling is set to quarantine.

Use d for days, m for months, y for years. The minimum value is 1d (one day).

The default value is 1y (one year).

Drive workers

Set the number of workers per drive used for fresh drive healing. Fresh drive healing populates replacement or newly added drives with data from existing erasure set members.

When set to 0, MinIO AIStor automatically sets this to GOMAXPROCS/2.

The default value is 0 (automatic).

Auto healing

Enable or disable automatic healing of freshly connected or replaced drives. When enabled, MinIO AIStor detects new or recovered drives and automatically initiates healing to restore data.

Valid values are on or off.

The default value is on.

Background workers

Set the number of background heal workers per node. Background healing repairs objects identified by the scanner or the Most Recently Failed (MRF) queue.

When set to 0, MinIO AIStor automatically sets this to GOMAXPROCS/2.

The default value is 0 (automatic).