Bucket-level Quality of Service

AIStor includes a Quality of Service (QoS) configuration that allows you to define rate limiting and concurrency controls for S3 requests on a per-bucket basis. This feature enables fine-grained control over API request rates and concurrent operations to ensure optimal resource utilization and prevent system overload.

You can configure rules with QoS to do the following:

Feature Description
Rate Limiting Control the number of requests per second for specified operations
Concurrency Control Limit the number of concurrent operations
Prefix-based Targeting Apply limits to specific object prefixes within buckets
API-specific Rules Target specific S3 operations
Priority-based Evaluation Use rule priorities for complex scenarios

In production environments, you typically write a YAML configuration file for your QoS rules that looks like the following:

version: "v1" 
rules:
  - id: "upload-rate-limit"
    label: "Limit upload operations"
    priority: 1
    objectPrefix: "uploads/"
    api: "s3.PutObject"
    rate: 100
    burst: 20
    limit: "rps"

Then apply these rules with the following command:

mc qos rule import ALIAS/my-bucket < /path/to/qos-config.yaml

To update rules, edit the YAML file and run mc qos rule import again. The YAML is replaced, not appended, so make sure not to remove or change any existing rules you need to keep.

See also QoS Rule Examples.

You can also add individual rules with the mc qos rule add command:

mc qos rule add ALIAS/my-bucket/my-prefix  --api "s3.GetObject" --rate 20   --limit rps --burst 50 --priority 1

See also the reference documentation for mc qos rule.

You can also monitor operations configured with QoS with mc qos status

QoS rule syntax

Parameters

Parameter Type Description
objectPrefix string Required
Object prefix to match.
"" to match all objects. Wildcards are not allowed.
priority integer Required
Rule priority.
The lowest number has the highest priority.
api string Required
API operation name.
Must be a valid S3, SFTP, or administrative API name. Supports only the * wildcard character.
rate integer Required
Rate limit value. Must be greater than 0.
burst integer Required  if limit is "rps". Must be greater than 0.
Number of requests to temporarily allow above the rate.
id string Optional
Identifier for the rule.
Default value: a generated UUID.
Should be unique across all rules.
label string Optional
Human-readable description
limit string Optional
Limit type. One of "rps" or "concurrency".
Default value "rps"

The combination of objectPrefix and api must be unique across all rules.

Duplicate ID values
If you provide duplicate ID values, all rules with the same ID are applied, but if you call mc qos rule remove, all rules with the same ID are deleted.

Limit types

AIStor QoS supports the following limit types:

  • "rps" (Requests Per Second)

    Controls the rate of API requests using a token bucket algorithm.

    YAML example:

    limit: "rps"
rate: 100      # 100 requests per second
burst: 20      # Allow bursts up to 20 requests above the rate

  • "concurrency"

    Controls the maximum number of concurrent operations, and blocks new requests when rate is reached.

    YAML example:

    limit: "concurrency"
rate: 10       # Maximum 10 concurrent operations

Rule evaluation order

For each request, rules are evaluated in priority order, where the lowest number has the highest priority. AIStor applies the first matching rule based on:

  • Object prefix match
  • API operation match

Duplicate priority values for different combinations of object prefix, API operation, and limit are allowed. Duplicate combinations of object prefix, API operation, and limit fail with an error that describes the issue.

If no rules match, the request proceeds with no QoS limits.