Use the following checklist when tuning the hardware and operating system configuration for a production, distributed MinIO AIStor deployment.
These settings align with the health checks performed by SUBNET diagnostics.
Upload a diagnostic report with mc support diag ALIAS and review the results in SUBNET to validate your deployment against these recommendations.
Set to performance on all nodes. Other governors (such as powersave or ondemand) reduce throughput under load. See CPU tuning.
Vector extensions
Verify that SSE 4.2 and AVX2 extensions are available on all CPUs. MinIO AIStor uses these for erasure coding and hashing. Check with lscpu | grep -i -E 'sse4_2|avx2'.
Minimum cores
A minimum of 8 physical cores per node is recommended. Nodes with fewer cores may bottleneck under concurrent S3 workloads.
CPU consistency
All nodes in the cluster should have the same CPU make, model, and core count. Asymmetric CPU configurations can cause uneven performance.
Disable swap or set vm.swappiness=0. Swap causes unpredictable latency spikes. See Memory tuning.
Transparent huge pages
Set to madvise, not always. The always setting can cause latency spikes and memory fragmentation. See Memory tuning.
IOMMU
Disable IOMMU if not required for device passthrough. IOMMU adds overhead to DMA operations. Do not disable it on a host that serves S3 over RDMA into GPU memory, which needs PCIe peer-to-peer DMA. See Bootloader settings and Clear PCIe ACS redirect.
Memory consistency
All nodes in the cluster should have the same amount of RAM. Asymmetric memory configurations can cause uneven performance.
Set max_retries to a finite value instead of the default -1 (infinite). Infinite retries can cause processes to hang on failing drives. See Disable XFS retry on error.
Mount options
Mount XFS drives with noatime,nodiratime. This eliminates unnecessary access time updates on every read operation. See Storage requirements.
Drive usage
Keep drive usage below 80%. Performance degrades as drives fill due to XFS fragmentation and reduced free extent availability. Monitor with mc support diag ALIAS.
Drive consistency
All nodes should have the same number, type, and size of drives. Asymmetric drive configurations can cause uneven performance.
Maximize RX and TX ring buffer sizes on all network interfaces used by MinIO AIStor. Small ring buffers cause packet drops under load. See Ring buffers.
Internode latency
Internode network round-trip time should remain below 10 ms. High latency impacts distributed lock performance and erasure coding operations. See Network tuning.
On a deployment using inter-node RDMA over a RoCE fabric, enable PFC end to end, DCQCN on every host NIC, and ECN with RED on every switch in the path. A lossy fabric collapses under the all-to-all erasure-write pattern. See Configure the RDMA fabric.
Set the open file descriptor limit to at least 65536 for the MinIO AIStor process. Set LimitNOFILE=65536 in the systemd unit file or configure with ulimit -n 65536.
Process user
Run MinIO AIStor as a dedicated non-root user. Running as root is a security risk and is flagged by SUBNET diagnostics. See the installation guides for user creation steps.