QoS at the Edge

MinIO AIStor Quality of Service is enforced at two layers:

• In-cluster QoS — per-bucket / per-prefix / per-API request rate (rps) and concurrency controls, enforced inside the MinIO AIStor server. See Bucket-level Quality of Service.
• QoS at the edge — per-bucket / per-prefix bandwidth (bytes per second) and request-rate controls, enforced by minwall, the edge component of AIStor that runs in front of (or co-located with) each MinIO AIStor server.

The two layers stack and complement each other. This page covers the edge layer.

Most operators throttle requests per second (often referred to as IOPS). That can be enforced either in-cluster or at the edge depending on whether you want the request rejected before authentication and EC pipeline setup, or after. Bandwidth (bytes per second) is enforced only at the edge.

Throttling is only meaningful when it is applied before the cluster commits resources to a request. By the time a request reaches the MinIO AIStor process, the expensive work has already started or is imminent:

• The TCP connection and TLS session are established
• The request line and headers have been parsed
• The S3 signature (HMAC-SHA256, computed over the streamed body for streaming SigV4) is verified
• For PUT / POST, the body is streaming from the kernel socket buffer into the erasure-coded write pipeline, which fans out to N drives across the EC set
• For GET / HEAD, the object has been located, enough erasure shards have been decoded to begin streaming, and the response socket is filling

A cap installed after the connection is accepted can only slow the data path that is already running. It cannot recover the CPU spent on TLS and signature verification, the disk I/O that erasure coding has scheduled, or the network buffers that the kernel has already committed. In a clustered deployment, each storage node only sees its own slice of the traffic, so an in-process bandwidth cap cannot honestly enforce a cluster-wide per-bucket budget without inter-node coordination.

The edge layer addresses both problems:

• It terminates the client connection and reads the API verb and bucket from the request, then paces the body in both directions against a per-bucket token bucket:
  • For uploads, it reads from the client at the configured uploadRatePerSec. The MinIO AIStor server sees only the paced stream — it never has to handle a faster ingress than the rule allows.
  • For downloads, it paces the response bytes it forwards to the client at downloadRatePerSec. TCP back-pressure flows through the proxy pipe and stalls the backend, so the storage node does not keep producing data faster than the client is allowed to consume it.
• For request-rate rules, it counts the request against a per-bucket, per-API token bucket and returns 429 Too Many Requests before the request reaches MinIO AIStor — before authentication and EC pipeline setup — when the bucket is empty.
• Edge access rules (anonymous: "deny", access: "deny") reject matching traffic at the boundary, so the storage process pays no cost for traffic that was always going to be refused.

minwall release artifacts are published under dl.min.io/aistor/minwall/release/ for the two supported Linux architectures:

Each release directory contains the latest binary, the corresponding RELEASE.<timestamp> build, SHA-256 checksums, and minisign / GPG signatures. A .fips variant is also published for FIPS-validated builds.

Install the latest binary:

curl --progress-bar -L https://dl.min.io/aistor/minwall/release/linux-amd64/minwall -o minwall
chmod +x ./minwall
sudo mv ./minwall /usr/local/bin/

curl --progress-bar -L https://dl.min.io/aistor/minwall/release/linux-arm64/minwall -o minwall
chmod +x ./minwall
sudo mv ./minwall /usr/local/bin/

Verify the download against minwall.sha256sum from the same release directory before installing on production hosts.

The edge layer is a reverse proxy. Clients connect to it (S3, SFTP, Console, or health endpoints) and it forwards to one or more MinIO AIStor backends using a least-connections strategy with optional local affinity.

The recommended topology is to co-locate minwall with each MinIO AIStor server on the same host. With local affinity set, minwall sends up to affinity concurrent requests to the MinIO AIStor server on the same node before spilling over to peer backends, which keeps short-lived requests on the loopback while preserving cluster-wide failover. In this topology, the byte-pacing for bandwidth rules happens entirely within the local minwall process — the MinIO AIStor process on the same node sees only the paced byte stream.

Bucket rules are declared under minio.rules.buckets[]. Bucket-specific or prefix-specific rules take precedence over the wildcard * rule. If a bucket rule omits a field (for example, bandwidth), the wildcard rule’s value applies.

The most common use case. The following rule caps s3.PutObject on mybucket at 100 rps with a burst of 100, and s3.ListObjectsV2 at 10 rps with a burst of 10. Other APIs are not limited by this rule.

version: "v1"
minio:
  endpoint:
    s3: "http://minio{1...4}:9000"
  rules:
    buckets:
      - bucket: "mybucket"
        label: "rate-limit-mybucket"
        requests:
          - api: "s3.PutObject"
            rate: 100
            burst: 100
          - api: "s3.ListObjectsV2"
            rate: 10
            burst: 10

When the bucket is empty, the edge layer returns 429 Too Many Requests before the request reaches the MinIO AIStor backend.

Cap both upload and download bandwidth at 4 MiB/s for any request to bucket:

version: "v1"
minio:
  endpoint:
    s3: "http://minio{1...4}:9000"
  rules:
    buckets:
      - bucket: "bucket"
        label: "bandwidth-throttle-bucket"
        bandwidth:
          downloadRatePerSec: "4MiB"
          uploadRatePerSec: "4MiB"

Throttle mybucket* at 10 MB/s and 10 rps per write API, then apply a 1 MB/s and 100 rps default to every other bucket through the * wildcard rule:

version: "v1"
minio:
  endpoint:
    s3: "http://minio{1...4}:9000"
  rules:
    buckets:
      - bucket: "mybucket*"
        label: "throttle-mybucket"
        bandwidth:
          downloadRatePerSec: "10MB"
          uploadRatePerSec: "10MB"
        requests:
          - api: "s3.PutObject"
            rate: 10
            burst: 10
          - api: "s3.ListObjectsV2"
            rate: 10
            burst: 10
      - bucket: "*"
        label: "default-rule"
        bandwidth:
          downloadRatePerSec: "1MB"
          uploadRatePerSec: "1MB"
        requests:
          - api: "s3.GetObject"
            rate: 100
            burst: 100
          - api: "s3.PutObject"
            rate: 100
            burst: 100

A minimal config that places the edge layer in front of a 4-node cluster, terminates S3 on :8000, and prefers the local MinIO AIStor instance for up to 100 concurrent requests before load-balancing to peer nodes:

version: "v1"
server:
  address:
    s3: ":8000"
    health: ":8080"
minio:
  affinity: 100
  endpoint:
    s3: "http://minio{1...4}:9000"
  health:
    path: "/minio/health/ready"
    interval: "10s"
    timeout: "10s"
  rules:
    buckets:
      - bucket: "*"
        label: "global-defaults"
        bandwidth:
          downloadRatePerSec: "100MiB"
          uploadRatePerSec: "100MiB"
        requests:
          - api: "s3.PutObject"
            rate: 500
            burst: 500
          - api: "s3.GetObject"
            rate: 500
            burst: 500

Point S3 clients at the minwall endpoint (:8000 in this example) rather than at MinIO AIStor directly. External load-balancer health probes should target the :8080 endpoint.

A typical production deployment uses both layers: the edge layer enforces bandwidth and rejects egregious traffic patterns before they consume backend CPU, while in-cluster QoS protects internal resources from contention even when traffic is well-behaved at the edge.

• Bucket-level Quality of Service — in-cluster request-rate and concurrency controls.
• QoS Rule Examples
• Supported APIs for QoS