Erasure Code Settings

This page covers settings that configure the Erasure Code parity to use for objects written to the AIStor cluster. This impacts how AIStor uses the space on the drive(s) and how AIStor can recover objects stored on lost drives or similar issues.

You can establish or modify settings by defining:

  • an environment variable on the host system prior to starting or restarting the AIStor Server. Refer to your operating system’s documentation for how to define an environment variable.
  • a configuration setting using mc admin config set.

If you define both an environment variable and the similar configuration setting, AIStor uses the environment variable value.

Some settings have only an environment variable or a configuration setting, but not both.

Each configuration setting controls fundamental AIStor behavior and functionality. Test configuration changes in a lower environment, such as DEV or QA, before applying to production.

Standard Storage Class

AIStor ‘storage classes’ have distinct functionality for AWS S3 storage classes. In context of AIStor, changing storage classes affects the erasure code parity of the deployment.

The parity level for the deployment. AIStor shards objects written with the default STANDARD storage class using this parity value.

AIStor references the x-amz-storage-class header in request metadata for determining which storage class to assign an object. The specific syntax or method for setting headers depends on your preferred method for interfacing with the AIStor server.

Specify the value using EC:M notation, where M refers to the number of parity blocks to create for the object.

The following table lists the default values based on the erasure set size of the initial server pool in the deployment:

Erasure Set Size Default Parity (EC:N)
1 EC:0
2-3 EC:1
4-5 EC:2
6 - 7 EC:3
8 - 16 EC:4

The minimum supported value is 0, which indicates no erasure coding protections. These deployments rely entirely on the storage controller or resource for availability / resiliency.

The maximum value depends on the erasure set size of the initial server pool in the deployment, where the upper bound is ERASURE_SET_SIZE/2. For example, a deployment with erasure set stripe size of 16 has a maximum standard parity of 8.

You can change this value after startup to any value between 0 and the upper bound for the erasure set size. AIStor only applies the changed parity to newly written objects. Existing objects retain the parity value in place at the time of their creation.

Reduced Redundancy Storage Class

The parity level for objects written with the REDUCED storage class.

AIStor references the x-amz-storage-class header in request metadata for determining which storage class to assign an object. The specific syntax or method for setting headers depends on your preferred method for interfacing with the AIStor server.

Specify the value using EC:M notation, where M refers to the number of parity blocks to create for the object.

This value must be less than or equal to MINIO_STORAGE_CLASS_STANDARD.

You cannot set this value for deployments with an erasure set size less than 2. Defaults to EC:1 for deployments with erasure set size greater than 1. Defaults to EC:0 for deployments of erasure set size of 1.

Parity Retention Optimization

AIStor by default automatically “upgrades” parity for an object if the destination erasure set maintains write quorum but has one or more drives offline. This behavior helps ensure that the given object maintains the same availability as objects written to the healthy erasure set.

Specify capacity to this setting to direct AIStor to not create any additional parity for the object. This prioritizes the overall capacity of the cluster at the cost of potentially reduced object availability in the event more drives in that erasure set fail.

Storage Class Comment

Adds a comment to the storage class settings.

All rights reserved 2024-Present, MinIO, Inc.