Table of Contents
MinIO supports publishing bucket notification events to PostgreSQL. MinIO supports PostgreSQL 9.5 and later only.
The following procedure adds a new PostgreSQL service endpoint for supporting bucket notifications in a MinIO deployment.
MinIO relies on features introduced with PostgreSQL 9.5.
mc
Command Line ToolThis procedure uses the mc
command line tool for certain actions.
See the mc
Quickstart for installation instructions.
You can configure a new PostgreSQL service endpoint using either environment variables or by setting runtime configuration settings.
MinIO supports specifying the PostgreSQL service endpoint and associated
configuration settings using
environment variables. The
minio server
process applies the specified settings on its
next startup.
The following example code sets all environment variables related to configuring a PostgreSQL service endpoint. The minimum required variables are:
set MINIO_NOTIFY_CONNECTION_STRING_<IDENTIFIER>="on"
set MINIO_NOTIFY_TABLE_<IDENTIFIER>="ENDPOINT"
set MINIO_NOTIFY_FORMAT_<IDENTIFIER>=""
set MINIO_NOTIFY_MAX_OPEN_CONNECTIONS_<IDENTIFIER>=""
set MINIO_NOTIFY_QUEUE_DIR_<IDENTIFIER>=""
set MINIO_NOTIFY_QUEUE_LIMIT_<IDENTIFIER>=""
set MINIO_NOTIFY_COMMENT_<IDENTIFIER>=""
Replace <IDENTIFIER>
with a unique descriptive string for the
PostgreSQL service endpoint. Use the same <IDENTIFIER>
value for all
environment variables related to the new target service endpoint.
The following examples assume an identifier of PRIMARY
.
If the specified <IDENTIFIER>
matches an existing PostgreSQL service
endpoint on the MinIO deployment, the new settings override
any existing settings for that endpoint. Use
mc admin config get notify_postgresql
to
review the currently configured PostgreSQL endpoints on the MinIO deployment.
Replace <ENDPOINT>
with the PostgreSQL Connection String
for PostgreSQL service endpoint. MinIO supports key=value
format for
the connection string. For example:
"host=https://postgresql.example.com port=5432 ..."
For more complete documentation on supported PostgreSQL connection string parameters, see PostgreSQL Connection String.
See PostgreSQL Service for Bucket Notifications for complete documentation on each environment variable.
MinIO supports adding or updating PostgreSQL endpoints on a running
minio server
process using the mc admin config set
command
and the notify_postgresql
configuration key. You must restart the
minio server
process to apply any new or updated configuration
settings.
The following example code sets all settings related to configuring an PostgreSQL service endpoint. The minimum required setting are:
mc admin config set ALIAS/ notify_postgresql:IDENTIFIER \
connection_string="ENDPOINT" \
table="<string>" \
format="<string>" \
max_open_connections="<string>" \
queue_dir="<string>" \
queue_limit="<string>" \
comment="<string>"
Replace IDENTIFIER
with a unique descriptive string for the
PostgreSQL service endpoint. The following examples in this procedure
assume an identifier of PRIMARY
.
If the specified IDENTIFIER
matches an existing PostgreSQL service
endpoint on the MinIO deployment, the new settings override
any existing settings for that endpoint. Use
mc admin config get notify_postgresql
to
review the currently configured PostgreSQL endpoints on the MinIO deployment.
Replace <ENDPOINT>
with the PostgreSQL URI connection string
of the PostgreSQL service endpoint. MinIO supports key=value
format
for the PostgreSQL connection string. For example:
"host=https://postgresql.example.com port=5432 ..."
For more complete documentation on supported PostgreSQL connection string parameters, see PostgreSQL Connection String.
See PostgreSQL Bucket Notification Configuration Settings for complete documentation on each setting.
You must restart the MinIO deployment to apply the configuration changes.
Use the mc admin service restart
command to restart the deployment.
mc admin service restart ALIAS
Replace ALIAS
with the alias of the deployment to
restart.
The minio server
process prints a line on startup for each configured PostgreSQL
target similar to the following:
SQS ARNs: arn:minio:sqs::primary:postgresql
You must specify the ARN resource when configuring bucket notifications with the associated PostgreSQL deployment as a target.
Identifying the ARN for your bucket notifications
You defined the <IDENTIFIER>
to assign to the target ARN for your bucket notifications when creating the endpoint previously.
The steps below return the ARNs configured on the deployment.
Identify the ARN created previously by looking for the <IDENTIFIER>
you specified.
Review the JSON output
Copy and run the following command, replacing ALIAS
with the alias of the deployment.
mc admin info --json ALIAS
In the JSON output, look for the key info.sqsARN
.
The ARN you need is the value of that key that matches the <IDENTIFIER>
you specified.
For example, arn:minio:sqs::primary:postgresql
.
Use jq to parse the JSON for the value
Copy and run the following command, replacing ALIAS
with the alias of the deployment.
mc admin info --json ALIAS | jq .info.sqsARN
This returns the ARN to use for notifications, such as arn:minio:sqs::primary:postgresql
Use the mc event add
command to add a new bucket notification
event with the configured PostgreSQL service as a target:
mc event add ALIAS/BUCKET arn:minio:sqs::primary:postgresql \
--event EVENTS
Replace ALIAS
with the alias of a MinIO deployment.
Replace BUCKET
with the name of the bucket in which to configure the ßevent.
Replace EVENTS
with a comma-separated list of events for which MinIO triggers notifications.
Use mc event list
to view all configured bucket events for
a given notification target:
mc event list ALIAS/BUCKET arn:minio:sqs::primary:postgresql
Perform an action on the bucket for which you configured the new event and
check the PostgreSQL service for the notification data. The action required
depends on which events
were specified
when configuring the bucket notification.
For example, if the bucket notification configuration includes the
s3:ObjectCreated:Put
event, you can use the
mc cp
command to create a new object in the bucket and trigger
a notification.
mc cp ~/data/new-object.txt ALIAS/BUCKET
The following procedure updates an existing PostgreSQL service endpoint for supporting bucket notifications in a MinIO deployment.
MinIO relies on features introduced with PostgreSQL 9.5.
mc
Command Line ToolThis procedure uses the mc
command line tool for certain actions.
See the mc
Quickstart for installation instructions.
Use the mc admin config get
command to list the currently
configured PostgreSQL service endpoints in the deployment:
mc admin config get ALIAS/ notify_postgresql
Replace ALIAS
with the alias of the MinIO deployment.
The command output resembles the following:
notify_postgresql:primary queue_dir="" connection_string="postgresql://" queue_limit="0" table="" format="namespace"
notify_postgresql:secondary queue_dir="" connection_string="" queue_limit="0" table="" format="namespace"
The notify_postgresql
key is the top-level configuration key for an
PostgreSQL Service for Bucket Notifications. The
connection_string
key specifies
the PostgreSQL service endpoint for the given notify_postgresql key. The
notify_postgresql:<IDENTIFIER>
suffix describes the unique identifier for
that PostgreSQL service endpoint.
Note the identifier for the PostgreSQL service endpoint you want to update for the next step.
Use the mc admin config set
command to set the new configuration
for the PostgreSQL service endpoint:
mc admin config set ALIAS/ notify_postgresql:IDENTIFIER \
connection_string="ENDPOINT" \
table="<string>" \
format="<string>" \
max_open_connections="<string>" \
queue_dir="<string>" \
queue_limit="<string>" \
comment="<string>"
The following configuration settings are the minimum required for a PostgreSQL service endpoint:
All other configuration settings are optional. See PostgreSQL Service for Bucket Notifications for a complete list of PostgreSQL configuration settings.
You must restart the MinIO deployment to apply the configuration changes.
Use the mc admin service restart
command to restart the deployment.
mc admin service restart ALIAS
Replace ALIAS
with the alias of the deployment to
restart.
The minio server
process prints a line on startup for each configured PostgreSQL
target similar to the following:
SQS ARNs: arn:minio:sqs::primary:postgresql
Perform an action on a bucket which has an event configuration using the updated
PostgreSQL service endpoint and check the PostgreSQL service for the notification data. The
action required depends on which events
were
specified when configuring the bucket notification.
For example, if the bucket notification configuration includes the
s3:ObjectCreated:Put
event, you can use the
mc cp
command to create a new object in the bucket and trigger
a notification.
mc cp ~/data/new-object.txt ALIAS/BUCKET
This work is licensed under a Creative Commons Attribution 4.0 International License.
©2020-Present, MinIO, Inc.