Bucket Notifications
AIStor bucket notifications allow administrators to send notifications to supported external services when a events occurs on either an object or bucket. AIStor server supports a range of possible events, listed below, that can trigger a notification. AIStor supports bucket and object-level S3 events similar to the Amazon S3 Event Notifications.
Supported notification targets
AIStor supports publishing event notifications to the following targets:
Target | Description |
---|---|
AMQP (RabbitMQ) | Publish notifications to an AMQP service such as RabbitMQ. See Publish Events to AMQP (RabbitMQ) for a tutorial. |
MQTT | Publish notifications to an MQTT service. See Publish Events to MQTT for a tutorial. |
NATS | Publish notifications to a NATS service. See Publish Events to NATS for a tutorial. |
NSQ | Publish notifications to a NSQ service. See Publish Events to NSQ for a tutorial |
Elasticsearch | Publish notifications to a Elasticsearch service. See Publish Events to Elasticsearch for a tutorial. |
Kafka | Publish notifications to a Kafka service. See Publish Events to Kafka for a tutorial. |
MySQL | Publish notifications to a MySQL service. See Publish Events to MySQL for a tutorial. |
PostgreSQL | Publish notifications to a PostgreSQL service. See Publish Events to PostgreSQL for a tutorial. |
Redis | Publish notifications to a Redis service. See Publish Events to Redis for a tutorial. |
webhook | Publish notifications to a Webhook service. See Publish Events to Webhook for a tutorial. |
Asynchronous vs synchronous bucket notifications
AIStor supports either asynchronous (default) or synchronous bucket notifications for all remote targets.
With asynchronous delivery, AIStor sends the event to the configured remote and does not wait for a response before continuing to the next event. Asynchronous bucket notification prioritizes sending events with the risk of some events being lost if the remote target has a transient issue during transit or processing.
With synchronous delivery, AIStor sends the event to the configured remote and then waits for the remote to confirm a successful receipt before continuing to the next event. Synchronous bucket notification prioritizes delivery of events with the risk of a slower event-send rate and queue fill.
To enable synchronous bucket notifications for all configured remote targets, use either of the following settings:
- Set the
MINIO_API_SYNC_EVENTS
environment variable toon
and restart the Object Store. - Set the
api.sync_events
configuration setting toon
and restart the Object Store.
For synchronous and asynchronous events, AIStor maintains a per-remote queue where it stores unsent and pending events.
The queue limit defaults to 100000
.
AIStor discards new events when the queue is full.
You can increase the queue size as necessary to better accommodate the rate of event send and processing of the AIStor deployment and remote target.
Use the QUEUE_LIMIT
environment variable or configuration setting for your notification method to modify this limit.
For asynchronous events, AIStor allows a maximum of 50000
concurrent send
calls.
Supported S3 event types
AIStor bucket notifications are compatible with Amazon S3 Event Notifications. This section lists all supported events.
Object Events
AIStor supports triggering notifications on the following S3 object events:
Object Accessed
s3:ObjectAccessed:Get
s3:ObjectAccessed:GetLegalHold
s3:ObjectAccessed:GetRetention
s3:ObjectAccessed:Head
s3:ObjectAccessed:\*
selects all s3:ObjectAccessed
-prefixed events.
Object created
s3:ObjectCreated:CompleteMultipartUpload
s3:ObjectCreated:Copy
s3:ObjectCreated:DeleteTagging
s3:ObjectCreated:Post
s3:ObjectCreated:Put
s3:ObjectCreated:PutLegalHold
s3:ObjectCreated:PutRetention
s3:ObjectCreated:PutTagging
s3:ObjectCreated:\*
selects all s3:ObjectCreated
-prefixed events.
Object removed
s3:ObjectRemoved:Delete
s3:ObjectRemoved:DeleteMarkerCreated
s3:ObjectRemoved:\*
selects all s3:ObjectRemoved
-prefixed events.
Replication events
AIStor supports triggering notifications on the following S3 replication events:
s3:Replication:OperationCompletedReplication
s3:Replication:OperationFailedReplication
s3:Replication:OperationMissedThreshold
s3:Replication:OperationNotTracked
s3:Replication:OperationReplicatedAfterThreshold
s3:Replication:\*
elects all s3:Replication
-prefixed events.
ILM transition events
AIStor supports triggering notifications on the following S3 ILM transition events:
s3:ObjectRestore:Post
s3:ObjectRestore:Completed
s3:ObjectRestore:\*
selects all s3:ObjectRestore
-prefixed events.
s3:ObjectTransition:Failed
s3:ObjectTransition:Complete
s3:ObjectTransition:\*
selects all s3:ObjectTransition
-prefixed events.
Scanner Events
AIStor supports triggering notifications on the following S3 scanner transition events:
-
s3:Scanner:ManyVersions
Finds objects with more than 1,000 versions.
-
s3:Scanner:BigPrefix
Finds prefixes with more than 50,000 sub-folders.
Global Events
AIStor supports triggering notifications on the following global events. You can only listen to these events through the ListenNotification API:
s3:BucketCreated
s3:BucketRemoved
Payload schema
All notification payloads use the same overall schema. Depending on the type of notification, some fields may be omitted or have null values.
{
"eventVersion": "string",
"eventSource": "string",
"awsRegion": "string",
"eventTime": "string",
"eventName": "string",
"userIdentity": {
"principalId": "string"
},
"requestParameters": {
"key": "value"
},
"responseElements": {
"key": "value"
},
"s3": {
"s3SchemaVersion": "string",
"configurationId": "string",
"bucket": {
"name": "string",
"ownerIdentity": {
"principalId": "string"
},
"arn": "string"
},
"object": {
"key": "string",
"size": 10000,
"eTag": "string",
"contentType": "string",
"userMetadata": {
"key": "string"
},
"versionId": "string",
"sequencer": "string"
}
},
"source": {
"host": "string",
"port": "string",
"userAgent": "string"
}
}
Example
The following example is a notification for an s3:ObjectCreated:Put
event:
{
"EventName": "s3:ObjectCreated:Put",
"Key": "test-bucket/image.jpg",
"Records": [
{
"eventVersion": "2.0",
"eventSource": "minio:s3",
"awsRegion": "",
"eventTime": "2025-02-06T01:04:31.998Z",
"eventName": "s3:ObjectCreated:Put",
"userIdentity": {
"principalId": "access_key"
},
"requestParameters": {
"principalId": "access_key",
"region": "",
"sourceIPAddress": "192.168.1.10"
},
"responseElements": {
"x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
"x-amz-request-id": "182178E8B36AC9DF",
"x-minio-deployment-id": "2369dcb4-348b-4d30-8fc9-61ab089ba4bc",
"x-minio-origin-endpoint": "https://aistor.test.svc.cluster.local"
},
"s3": {
"s3SchemaVersion": "1.0",
"configurationId": "Config",
"bucket": {
"name": "test-bucket",
"ownerIdentity": {
"principalId": "access_key"
},
"arn": "arn:aws:s3:::test-bucket"
},
"object": {
"key": "image.jpg",
"size": 84452,
"eTag": "eb52f8e46f60a27a8a1a704e25757f30",
"contentType": "image/jpeg",
"userMetadata": {
"content-type": "image/jpeg"
},
"sequencer": "182178E8B3728CAC"
}
},
"source": {
"host": "192.168.1.10",
"port": "",
"userAgent": "MinIO (linux; amd64) minio-go/v7.0.83"
}
}
]
}