Network QoS

MinIO AIStor Quality of Service is enforced at two layers:

• API QoS — per-bucket / per-prefix / per-API request rate (rps) and concurrency controls, enforced inside the MinIO AIStor server. See Bucket-level Quality of Service.
• Network QoS — per-bucket / per-prefix and per-access-key bandwidth (bytes per second) and request-rate controls, enforced by minwall, the network-layer component of AIStor that runs in front of (or co-located with) each MinIO AIStor server.

The two layers stack and complement each other. This page covers Network QoS.

Most operators throttle requests per second (often referred to as IOPS). That can be enforced either by API QoS or by Network QoS depending on whether you want the request rejected before authentication and EC pipeline setup, or after. Bandwidth (bytes per second) is enforced only by Network QoS.

Throttling is only meaningful when it is applied before the cluster commits resources to a request. By the time a request reaches the MinIO AIStor process, the expensive work has already started or is imminent:

• The TCP connection and TLS session are established
• The request line and headers have been parsed
• The S3 signature (HMAC-SHA256, computed over the streamed body for streaming SigV4) is verified
• For PUT / POST, the body is streaming from the kernel socket buffer into the erasure-coded write pipeline, which fans out to N drives across the EC set
• For GET / HEAD, the object has been located, enough erasure shards have been decoded to begin streaming, and the response socket is filling

A cap installed after the connection is accepted can only slow the data path that is already running. It cannot recover the CPU spent on TLS and signature verification, the disk I/O that erasure coding has scheduled, or the network buffers that the kernel has already committed. In a clustered deployment, each storage node only sees its own slice of the traffic, so an in-process bandwidth cap cannot honestly enforce a cluster-wide per-bucket budget without inter-node coordination.

Network QoS addresses both problems:

• It terminates the client connection and reads the API verb and bucket from the request, then paces the body in both directions against a per-bucket token bucket:
  • For uploads, it reads from the client at the configured uploadRatePerSec. The MinIO AIStor server sees only the paced stream — it never has to handle a faster ingress than the rule allows.
  • For downloads, it paces the response bytes it forwards to the client at downloadRatePerSec. TCP back-pressure flows through the proxy pipe and stalls the backend, so the storage node does not keep producing data faster than the client is allowed to consume it.
• For request-rate rules, it counts the request against a per-bucket, per-API token bucket and returns 429 Too Many Requests before the request reaches MinIO AIStor — before authentication and EC pipeline setup — when the bucket is empty.
• Network access rules (anonymous: "deny", access: "deny") reject matching traffic at the boundary, so the storage process pays no cost for traffic that was always going to be refused.

minwall release artifacts are published under dl.min.io/aistor/minwall/release/ for the two supported Linux architectures:

Each release directory contains the latest binary, the corresponding RELEASE.<timestamp> build, SHA-256 checksums, and minisign / GPG signatures. A .fips variant is also published for FIPS-validated builds.

Install the latest binary:

curl --progress-bar -L https://dl.min.io/aistor/minwall/release/linux-amd64/minwall -o minwall
chmod +x ./minwall
sudo mv ./minwall /usr/local/bin/

curl --progress-bar -L https://dl.min.io/aistor/minwall/release/linux-arm64/minwall -o minwall
chmod +x ./minwall
sudo mv ./minwall /usr/local/bin/

Verify the download against minwall.sha256sum from the same release directory before installing on production hosts.

Network QoS runs as a reverse proxy. Clients connect to it (S3, SFTP, Console, or health endpoints) and it forwards to one or more MinIO AIStor backends using a least-connections strategy with optional local affinity.

The recommended topology is to co-locate minwall with each MinIO AIStor server on the same host. With local affinity set, minwall sends up to affinity concurrent requests to the MinIO AIStor server on the same node before spilling over to peer backends, which keeps short-lived requests on the loopback while preserving cluster-wide failover. In this topology, the byte-pacing for bandwidth rules happens entirely within the local minwall process — the MinIO AIStor process on the same node sees only the paced byte stream.

Network QoS supports two independent rule blocks. Bucket rules gate traffic by object path; access-key rules gate it by authenticated identity. Both are evaluated on every request and a request must pass both. Either block can be used on its own, or they can be combined.

Bucket rules are declared under minio.rules.buckets[]. Bucket-specific or prefix-specific rules take precedence over the wildcard * rule. If a bucket rule omits a field (for example, bandwidth), the wildcard rule’s value applies.

Access-key rules are declared under minio.rules.accessKeys[]. They apply per authenticated identity, evaluated in addition to any matching bucket rule. The access key is extracted from the SigV4 Authorization header for signed requests, or from X-Amz-Credential for presigned URLs. Anonymous requests are not affected by access-key rules; control unauthenticated access through rules.anonymous and per-bucket anonymous overrides.

Patterns are matched in the order listed, so place specific entries before wildcards: an exact key for the most specific rule, glob patterns such as team-* for groups of keys, and a literal "*" entry as a catch-all default.

The most common use case. The following rule caps s3.PutObject on mybucket at 100 rps with a burst of 100, and s3.ListObjectsV2 at 10 rps with a burst of 10. Other APIs are not limited by this rule.

version: "v1"
minio:
  endpoint:
    s3: "http://minio{1...4}:9000"
  rules:
    buckets:
      - bucket: "mybucket"
        label: "rate-limit-mybucket"
        requests:
          - api: "s3.PutObject"
            rate: 100
            burst: 100
          - api: "s3.ListObjectsV2"
            rate: 10
            burst: 10

When the bucket is empty, Network QoS returns 429 Too Many Requests before the request reaches the MinIO AIStor backend.

Cap both upload and download bandwidth at 4 MiB/s for any request to bucket:

version: "v1"
minio:
  endpoint:
    s3: "http://minio{1...4}:9000"
  rules:
    buckets:
      - bucket: "bucket"
        label: "bandwidth-throttle-bucket"
        bandwidth:
          downloadRatePerSec: "4MiB"
          uploadRatePerSec: "4MiB"

Throttle mybucket* at 10 MB/s and 10 rps per write API, then apply a 1 MB/s and 100 rps default to every other bucket through the * wildcard rule:

version: "v1"
minio:
  endpoint:
    s3: "http://minio{1...4}:9000"
  rules:
    buckets:
      - bucket: "mybucket*"
        label: "throttle-mybucket"
        bandwidth:
          downloadRatePerSec: "10MB"
          uploadRatePerSec: "10MB"
        requests:
          - api: "s3.PutObject"
            rate: 10
            burst: 10
          - api: "s3.ListObjectsV2"
            rate: 10
            burst: 10
      - bucket: "*"
        label: "default-rule"
        bandwidth:
          downloadRatePerSec: "1MB"
          uploadRatePerSec: "1MB"
        requests:
          - api: "s3.GetObject"
            rate: 100
            burst: 100
          - api: "s3.PutObject"
            rate: 100
            burst: 100

Revoke a compromised key at the network boundary, cap an ingest service account at 200 MiB/s upload and 500 s3.PutObject rps, share a 50 MiB/s budget across every key in the team-* group, and apply a 10 MiB/s default to anyone else:

version: "v1"
minio:
  endpoint:
    s3: "http://minio{1...4}:9000"
  rules:
    accessKeys:
      - accessKey: "AKIA-REVOKED"
        label: "deny-compromised-key"
        access: "deny"
      - accessKey: "AKIA-INGEST"
        label: "ingest-key-bandwidth"
        bandwidth:
          uploadRatePerSec:   "200MiB"
          downloadRatePerSec: "50MiB"
        requests:
          - api: "s3.PutObject"
            rate: 500
            burst: 1000
      - accessKey: "team-*"
        label: "team-shared-limits"
        bandwidth:
          uploadRatePerSec:   "50MiB"
          downloadRatePerSec: "50MiB"
      - accessKey: "*"
        label: "default-per-key-limits"
        bandwidth:
          uploadRatePerSec:   "10MiB"
          downloadRatePerSec: "10MiB"

The revoked key is rejected with 403 Access Denied before MinIO AIStor sees the request. The ingest key’s burst write traffic is paced to 200 MiB/s in the proxy regardless of which bucket it targets. Every other authenticated request falls through to the * catch-all.

Access-key rules and bucket rules can be combined in the same config — for example, a per-bucket 100 MB/s download cap with a stricter 10 MB/s per-key cap means individual keys cannot saturate the bucket budget even when several of them target the same data.

A minimal config that places Network QoS in front of a 4-node cluster, terminates S3 on :8000, and prefers the local MinIO AIStor instance for up to 100 concurrent requests before load-balancing to peer nodes:

version: "v1"
server:
  address:
    s3: ":8000"
    health: ":8080"
minio:
  affinity: 100
  endpoint:
    s3: "http://minio{1...4}:9000"
  health:
    path: "/minio/health/ready"
    interval: "10s"
    timeout: "10s"
  rules:
    buckets:
      - bucket: "*"
        label: "global-defaults"
        bandwidth:
          downloadRatePerSec: "100MiB"
          uploadRatePerSec: "100MiB"
        requests:
          - api: "s3.PutObject"
            rate: 500
            burst: 500
          - api: "s3.GetObject"
            rate: 500
            burst: 500

Point S3 clients at the minwall endpoint (:8000 in this example) rather than at MinIO AIStor directly. External load-balancer health probes should target the :8080 endpoint.

A typical production deployment uses both layers: Network QoS enforces bandwidth and rejects egregious traffic patterns before they consume backend CPU, while API QoS protects internal resources from contention even when traffic is well-behaved at the network layer.

• Bucket-level Quality of Service — API-layer request-rate and concurrency controls.
• QoS Rule Examples
• Supported APIs for QoS