Monitoring

MinIO KMS provides API endpoints for monitoring deployment health, status, and performance, as well as Prometheus-compatible metrics for integration with monitoring systems.

Health endpoints

Prometheus metrics

The /v1/health/metrics endpoint exposes metrics in OpenMetrics v1.0 format, compatible with Prometheus text format.

Each node reports only its own activity. Counters increase until the process restarts, at which point they reset to zero. Gauges report the value at the time of the scrape.

Scrape configuration

A Prometheus scrape job for MinIO KMS needs three settings:

• scheme: https, because MinIO KMS serves HTTPS only and has no plaintext HTTP endpoint.
• metrics_path: /v1/health/metrics, because MinIO KMS serves metrics only at that path. Prometheus scrapes /metrics by default, and MinIO KMS does not serve that path.
• ca_file, the CA certificate that issued the MinIO KMS server certificate, so that Prometheus can verify the server.

The metrics endpoint is unprivileged. Prometheus does not need a client certificate to scrape it, so the job needs no cert_file or key_file.

Add the following job to your Prometheus configuration:

scrape_configs:
  - job_name: minkms
    scheme: https
    metrics_path: /v1/health/metrics
    tls_config:
      ca_file: /path/to/ca.crt
    static_configs:
      - targets:
          - MINKMS_HOST_1:7373
          - MINKMS_HOST_2:7373
          - MINKMS_HOST_3:7373

Replace the targets with your MinIO KMS node addresses.

Metric names

Metric names follow the OpenMetrics conventions. A metric that has a unit carries the unit as a name suffix: durations end in _seconds and byte counts end in _bytes. The tables below list each metric family under the name MinIO KMS exposes.

A query in PromQL, the Prometheus query language, uses the time series name rather than the family name. The two names differ for three of the four metric types:

• Counter: the series carries a _total suffix. Query the http_request family as http_request_total.
• Histogram: each family exposes NAME_bucket series, one per bucket boundary plus +Inf, along with NAME_sum and NAME_count.
• Info: the series carries an _info suffix. Query the runtime family as runtime_info.
• Gauge: query the family name unchanged.

One metric departs from the unit-suffix rule. net_conn_duration records seconds but carries no _seconds suffix, so query it as net_conn_duration_bucket, net_conn_duration_sum, and net_conn_duration_count.

Available metrics

Client API:

Duration buckets are 0.001, 0.01, 0.05, 0.1, 0.25, 0.5, 1, 3, and 10 seconds. Size buckets are 1, 256, 512, 1024, 4096, 16384, 65536, 524288, and 1048576 bytes.

Internode RPC:

These metrics cover traffic between cluster nodes. On a single-node deployment they stay at zero.

Duration buckets are 0.001, 0.01, 0.05, 0.1, 0.25, 0.5, 1, 3, and 10 seconds. Size buckets are 1, 256, 512, 1024, 4096, 16384, 65536, and 524288 bytes.

The two io series of rpc_http_request_duration_seconds measure different spans. The io="recv" series records the time this node spends handling a request from a peer. The io="send" series records the round trip, from the moment this node sends the request until the peer’s response headers arrive, so it includes network time and the peer’s handling time.

Network connections:

Connection lifetime buckets are 0.01, 0.1, 0.5, 1, 5, 10, 30, 60, 300, and 3600 seconds.

Consensus:

TLS:

MinIO KMS refreshes tls_cert_expiry_seconds in the background, so the value follows a certificate rotation without a restart. Compare it against time() to get the remaining validity.

Runtime:

Label values

Key metrics to monitor

This table uses time series names, so the counters carry their _total suffix.