Each v2 endpoint returns all metrics for its category.
For example, scraping the following endpoint returns all cluster metrics:
http://HOSTNAME:PORT/minio/v2/metrics/cluster
The base endpoint alone, /minio/v2/metrics/, returns cluster metrics.
For more flexible scraping and a wider range of metrics, use version 3 metrics.
Existing deployments can continue to use version 2 metrics.
The following sections describe the version 2 endpoints and metrics.
Cluster metrics
AIStor collects the following v2 metrics at the cluster level.
Metrics may include one or more labels, such as the server that calculated that metric.
Cluster metrics can be scraped from the following URL:
https://HOSTNAME:PORT/minio/v2/metrics/cluster
Replace HOSTNAME:PORT with the hostname of your AIStor deployment.
For deployments behind a load balancer, use the load balancer hostname instead of a single node hostname.
Audit metrics
Name
Description
minio_audit_failed_messages
Total number of messages that failed to send since start.
minio_audit_target_queue_length
Number of unsent messages in queue for target.
minio_audit_total_messages
Total number of messages sent since start.
Batch metrics
Name
Description
minio_cluster_batch_catalog_jobs_completed
Number of completed batch catalog jobs.
minio_cluster_batch_catalog_jobs_failed
Number of failed batch catalog jobs.
minio_cluster_batch_catalog_jobs_in_progress
Number of in-progress batch catalog jobs.
minio_cluster_batch_expire_jobs_completed
Number of completed batch expire jobs.
minio_cluster_batch_expire_jobs_failed
Number of failed batch expire jobs.
minio_cluster_batch_expire_jobs_in_progress
Number of in-progress batch expire jobs.
minio_cluster_batch_keyrotate_jobs_completed
Number of completed batch keyrotate jobs.
minio_cluster_batch_keyrotate_jobs_failed
Number of failed batch keyrotate jobs.
minio_cluster_batch_keyrotate_jobs_in_progress
Number of in-progress batch keyrotate jobs.
minio_cluster_batch_replicate_jobs_completed
Number of completed batch replicate jobs.
minio_cluster_batch_replicate_jobs_failed
Number of failed batch replicate jobs.
minio_cluster_batch_replicate_jobs_in_progress
Number of in-progress batch replicate jobs.
Cluster capacity metrics
Name
Description
minio_cluster_capacity_raw_free_bytes
Total free capacity online in the cluster.
minio_cluster_capacity_raw_total_bytes
Total capacity online in the cluster.
minio_cluster_capacity_usable_free_bytes
Total free usable capacity online in the cluster.
minio_cluster_capacity_usable_total_bytes
Total usable capacity online in the cluster.
minio_cluster_objects_size_distribution
Distribution of object sizes across a cluster.
minio_cluster_objects_version_distribution
Distribution of object versions across a cluster.
minio_cluster_usage_object_total
Total number of objects in a cluster.
minio_cluster_usage_total_bytes
Total cluster usage in bytes.
minio_cluster_usage_version_total
Total number of versions (including delete markers) in a cluster.
minio_cluster_usage_deletemarker_total
Total number of delete markers in a cluster.
minio_cluster_bucket_total
Total number of buckets in the cluster.
Cluster drive metrics
Name
Description
minio_cluster_drive_offline_total
Total drives offline in this cluster.
minio_cluster_drive_online_total
Total drives online in this cluster.
minio_cluster_drive_total
Total drives in this cluster.
Cluster ILM metrics
Name
Description
minio_cluster_ilm_transitioned_bytes
Total bytes transitioned to a tier.
minio_cluster_ilm_transitioned_objects
Total number of objects transitioned to a tier.
minio_cluster_ilm_transitioned_versions
Total number of versions transitioned to a tier.
Cluster KMS metrics
Name
Description
minio_cluster_kms_online
Reports whether the KMS is online (1) or offline (0).
minio_cluster_kms_request_error
Number of KMS requests that failed due to some error. (HTTP 4xx status code).
minio_cluster_kms_request_failure
Number of KMS requests that failed due to some internal failure. (HTTP 5xx status code).
minio_cluster_kms_request_success
Number of KMS requests that succeeded.
minio_cluster_kms_uptime
The time the KMS has been up and running in seconds.
Cluster health metrics
Name
Description
minio_cluster_nodes_offline_total
Total number of AIStor nodes offline.
minio_cluster_nodes_online_total
Total number of AIStor nodes online.
minio_cluster_write_quorum
Maximum write quorum across all pools and sets.
minio_cluster_health_status
Get current cluster health status.
minio_cluster_health_erasure_set_healing_drives
Count of healing drives in the erasure set.
minio_cluster_health_erasure_set_online_drives
Count of online drives in the erasure set.
minio_cluster_health_erasure_set_read_quorum
Get read quorum of the erasure set.
minio_cluster_health_erasure_set_write_quorum
Get write quorum of the erasure set.
minio_cluster_health_erasure_set_status
Get current health status of the erasure set.
Cluster replication metrics
Cluster replication metrics are available for deployments that have configured site replication.
Name
Description
minio_cluster_replication_last_hour_failed_bytes
Total number of bytes failed at least once to replicate in the last full hour.
minio_cluster_replication_last_hour_failed_count
Total number of objects which failed replication in the last full hour.
Total action outcome of lifecycle checks since server start for deletion of all versions.
Tier metrics
Name
Description
minio_node_tier_tier_ttlb_seconds_distribution
Distribution of time to last byte for objects downloaded from warm tier.
minio_node_tier_requests_success
Number of requests to download object from warm tier that were successful.
minio_node_tier_requests_failure
Number of requests to download object from warm tier that were failure.
System metrics
Name
Description
minio_node_file_descriptor_limit_total
Limit on total number of open file descriptors for the AIStor Server process.
minio_node_file_descriptor_open_total
Total number of open file descriptors by the AIStor Server process.
minio_node_go_routine_total
Total number of go routines running.
minio_node_io_rchar_bytes
Total bytes read by the process from the underlying storage system including cache, /proc/[pid]/iorchar.
minio_node_io_read_bytes
Total bytes read by the process from the underlying storage system, /proc/[pid]/ioread_bytes.
minio_node_io_wchar_bytes
Total bytes written by the process to the underlying storage system including page cache, /proc/[pid]/iowchar.
minio_node_io_write_bytes
Total bytes written by the process to the underlying storage system, /proc/[pid]/iowrite_bytes.
minio_node_process_cpu_total_seconds
Total user and system CPU time spent in seconds by the process.
minio_node_process_resident_memory_bytes
Resident memory size in bytes.
minio_node_process_virtual_memory_bytes
Virtual memory size in bytes.
minio_node_process_starttime_seconds
Start time for AIStor process per node, time in seconds since Unix epoch.
minio_node_process_uptime_seconds
Uptime for AIStor process per node in seconds.
Scanner metrics
Name
Description
minio_node_scanner_bucket_scans_finished
Total number of bucket scans finished since server start.
minio_node_scanner_bucket_scans_started
Total number of bucket scans started since server start.
minio_node_scanner_directories_scanned
Total number of directories scanned since server start.
minio_node_scanner_objects_scanned
Total number of unique objects scanned since server start.
minio_node_scanner_versions_scanned
Total number of object versions scanned since server start.
minio_node_syscall_read_total
Total read SysCalls to the kernel. /proc/[pid]/iosyscr.
minio_node_syscall_write_total
Total write SysCalls to the kernel. /proc/[pid]/iosyscw.
minio_usage_last_activity_nano_seconds
Time elapsed (in nanoseconds) since last scan activity.
Bucket metrics
AIStor collects the following v2 metrics at the bucket level.
Each metric includes a bucket label to identify the corresponding bucket.
Metrics may include one or more additional labels, such as the server that calculated that metric.
The v2 bucket metrics endpoint only returns metrics for the first 100 buckets.
For more than 100 buckets, see metrics version 3.
Bucket metrics can be scraped from the following URL:
https://HOSTNAME:PORT/minio/v2/metrics/bucket
Replace HOSTNAME:PORT with the hostname of your AIStor deployment.
For deployments behind a load balancer, use the load balancer hostname instead of a single node hostname.
Batch metrics
Name
Description
minio_bucket_batch_catalog_objects
Number of completed batch job catalog objects.
minio_bucket_batch_catalog_objects_failed
Number of failed batch job catalog objects.
minio_bucket_batch_expire_objects
Number of completed batch job expire objects.
minio_bucket_batch_expire_objects_failed
Number of failed batch job expire objects.
minio_bucket_batch_keyrotate_objects
Number of completed batch job keyrotate objects.
minio_bucket_batch_keyrotate_objects_failed
Number of failed batch job keyrotate objects.
minio_bucket_batch_replicate_objects
Number of completed batch job replicate objects.
minio_bucket_batch_replicate_objects_failed
Number of failed batch job replicate objects.
Distribution metrics
Name
Description
minio_bucket_objects_size_distribution
Distribution of object sizes in the bucket, includes label for the bucket name.
minio_bucket_objects_version_distribution
Distribution of object sizes in a bucket, by number of versions.
Replication metrics
These metrics are available for deployments with bucket or batch replication.
Name
Description
minio_bucket_replication_last_minute_failed_bytes
Total number of bytes failed at least once to replicate in the last full minute.
minio_bucket_replication_last_minute_failed_count
Total number of objects which failed replication in the last full minute.
minio_bucket_replication_last_hour_failed_bytes
Total number of bytes failed at least once to replicate in the last full hour.
minio_bucket_replication_last_hour_failed_count
Total number of objects which failed replication in the last full hour.
minio_bucket_replication_total_failed_bytes
Total number of bytes failed at least once to replicate since server start.
minio_bucket_replication_total_failed_count
Total number of objects which failed replication since server start.
minio_bucket_replication_latency_ms
Replication latency in milliseconds.
minio_bucket_replication_received_bytes
Total number of bytes replicated to this bucket from another source bucket.
minio_bucket_replication_received_count
Total number of objects received by this bucket from another source bucket.
minio_bucket_replication_sent_bytes
Total number of bytes replicated to the target bucket.
minio_bucket_replication_sent_count
Total number of objects replicated to the target bucket.
minio_bucket_replication_credential_errors
Total number of replication credential errors since server start.
Number of failures in PUT tagging proxy requests to replication target.
Traffic metrics
Name
Description
minio_bucket_traffic_received_bytes
Total number of S3 bytes received for this bucket.
minio_bucket_traffic_sent_bytes
Total number of S3 bytes sent for this bucket.
Usage metrics
Name
Description
minio_bucket_usage_object_total
Total number of objects.
minio_bucket_usage_version_total
Total number of versions (including delete markers).
minio_bucket_usage_deletemarker_total
Total number of delete markers.
minio_bucket_usage_total_bytes
Total bucket size in bytes.
minio_bucket_quota_total_bytes
Total bucket quota size in bytes.
Requests metrics
Name
Description
minio_bucket_requests_4xx_errors_total
Total number of S3 requests with (4xx) errors on a bucket.
minio_bucket_requests_5xx_errors_total
Total number of S3 requests with (5xx) errors on a bucket.
minio_bucket_requests_inflight_total
Total number of S3 requests currently in flight on a bucket.
minio_bucket_requests_total
Total number of S3 requests on a bucket.
minio_bucket_requests_canceled_total
Total number S3 requests canceled by the client.
minio_bucket_requests_ttfb_seconds_distribution
Distribution of time to first byte across API calls per bucket.
Resource metrics
AIStor collects the following resource metrics at the node level.
Each metric includes a server label to identify the corresponding node.
Metrics may include one or more additional labels, such as the drive path, interface name, etc.
These metrics can be scraped from the following URL:
Replace HOSTNAME:PORT with the hostname of your AIStor deployment.
For deployments behind a load balancer, use the load balancer hostname instead of a single node hostname.
Drive resource metrics
Name
Description
minio_node_drive_total_bytes
Total storage capacity on a drive.
minio_node_drive_used_bytes
Total storage used on a drive.
minio_node_drive_total_inodes
Total inodes on a drive.
minio_node_drive_used_inodes
Total inodes used on a drive.
minio_node_drive_reads_per_sec
Reads per second on a drive.
minio_node_drive_reads_kb_per_sec
Kilobytes read per second on a drive.
minio_node_drive_reads_await
Average time for read requests to be served on a drive.
minio_node_drive_writes_per_sec
Writes per second on a drive.
minio_node_drive_writes_kb_per_sec
Kilobytes written per second on a drive.
minio_node_drive_writes_await
Average time for write requests to be served on a drive.
minio_node_drive_perc_util
Percentage of time the disk was busy since uptime.
Network interface metrics
Name
Description
minio_node_if_rx_bytes
Bytes received on the interface in 60s.
minio_node_if_rx_bytes_avg
Bytes received on the interface in 60s (avg) since uptime.
minio_node_if_rx_bytes_max
Bytes received on the interface in 60s (max) since uptime.