Connecting query engines to MinIO AIStor Tables

MinIO AIStor Tables exposes a native Iceberg REST Catalog so query and analytics engines can read and write Iceberg tables directly in MinIO AIStor object storage. This page consolidates connection details for the most common engines, a compatibility matrix, and complete, working catalog configurations.

The catalog is served from the /_iceberg base path on the MinIO AIStor S3 endpoint. For example, a cluster reachable at https://aistor.example.net:9000 serves its catalog at https://aistor.example.net:9000/_iceberg.

How authentication works

The MinIO AIStor Tables catalog requires AWS Signature Version 4 (SigV4) request signing with the signing (service) name s3tables. This is the single most important configuration detail: an engine must be able to sign catalog requests with SigV4 using s3tables as the signing name.

Engines that authenticate to the catalog only with OAuth2 bearer tokens cannot connect today, because the catalog does not accept bearer-token authentication.

The region value that an engine sends with SigV4 is required by the signing algorithm but is not otherwise used by MinIO AIStor Tables. Any non-empty value (for example, local or dummy) is acceptable.

For details on the underlying REST API and SigV4 headers, see the MinIO AIStor Tables API Reference. For policy-based access control of catalog actions, see Controlling access to MinIO AIStor Tables.

Supported-engine compatibility matrix

Engine Catalog authentication Supported today Notes
Apache Spark (Iceberg REST) SigV4 (s3tables) Yes Use the Iceberg REST catalog with rest.sigv4-enabled and rest.signing-name=s3tables.
PyIceberg SigV4 (s3tables) Yes Use rest.sigv4-enabled and rest.signing-name=s3tables.
Trino / Presto SigV4 (s3tables) Yes Property name for SigV4 differs by Trino version. See Trino.
Starburst SigV4 (s3tables) Yes Built on Trino. Use the Trino Iceberg REST connector with SigV4.
Dremio SigV4 (s3tables) Yes Connect through the Iceberg REST catalog with SigV4 signing.
ClickHouse OAuth2 bearer only No The ClickHouse Iceberg REST integration authenticates to the catalog with OAuth bearer tokens and cannot sign catalog requests with SigV4 today.
PuppyGraph OAuth2 bearer only No The PuppyGraph Iceberg REST integration authenticates to the catalog with OAuth bearer tokens and cannot sign catalog requests with SigV4 today.
“Not supported” refers to the catalog connection only. An engine listed as not supported can still read and write the underlying data and metadata files through the standard S3 API once a supported engine has registered them in the catalog. The limitation is specifically the inability to authenticate REST catalog requests with SigV4.

Shared configuration values

All examples use the following placeholder values. Replace them with values for your deployment:

Value Description
uri / catalog URI The MinIO AIStor S3 endpoint with the /_iceberg catalog path, for example https://aistor.example.net:9000/_iceberg.
warehouse The plain warehouse name (for example analytics). Do not prefix it with s3:// or s3a://.
s3.endpoint The MinIO AIStor S3 endpoint, for example https://aistor.example.net:9000.
region Required by SigV4 but unused by AIStor. Use any non-empty value such as local.
access key / secret key Credentials for a user with permission to access MinIO AIStor Tables.
Path-style access
Always enable path-style S3 access (s3.path-style-access=true and the equivalent Hadoop S3A setting). Virtual-host-style addressing is not used for warehouse buckets in these configurations.

PyIceberg

PyIceberg connects to the REST catalog with SigV4 signing. Install the dependencies with:

pip install pyiceberg pyarrow pandas

Load the catalog:

from pyiceberg.catalog import load_catalog

catalog = load_catalog(
    "aistor",
    **{
        "uri": "https://aistor.example.net:9000/_iceberg",
        "warehouse": "analytics",
        "rest.sigv4-enabled": "true",
        "rest.signing-name": "s3tables",
        "rest.signing-region": "local",   # required by SigV4, value unused
        "client.region": "local",
        "client.access-key-id": "YOUR-ACCESS-KEY",
        "client.secret-access-key": "YOUR-SECRET-KEY",
        "s3.endpoint": "https://aistor.example.net:9000",
        "s3.path-style-access": "true",
        "s3.access-key-id": "YOUR-ACCESS-KEY",
        "s3.secret-access-key": "YOUR-SECRET-KEY",
    }
)

For a complete end-to-end PyIceberg walkthrough that creates a warehouse, namespace, and table and then inserts and queries data, see MinIO AIStor Tables.

Spark

Spark uses the Iceberg Spark runtime with the REST catalog and SigV4 signing. The example below configures a catalog named aistor.

config = {
    # Catalog definition
    "spark.sql.catalog.aistor": "org.apache.iceberg.spark.SparkCatalog",
    "spark.sql.catalog.aistor.type": "rest",
    "spark.sql.catalog.aistor.uri": "https://aistor.example.net:9000/_iceberg",
    "spark.sql.catalog.aistor.warehouse": "analytics",

    # REST catalog SigV4 signing
    "spark.sql.catalog.aistor.rest.endpoint": "https://aistor.example.net:9000",
    "spark.sql.catalog.aistor.rest.access-key-id": "YOUR-ACCESS-KEY",
    "spark.sql.catalog.aistor.rest.secret-access-key": "YOUR-SECRET-KEY",
    "spark.sql.catalog.aistor.rest.sigv4-enabled": "true",
    "spark.sql.catalog.aistor.rest.signing-name": "s3tables",
    "spark.sql.catalog.aistor.rest.signing-region": "local",  # required, value unused

    # S3 data access
    "spark.sql.catalog.aistor.s3.endpoint": "https://aistor.example.net:9000",
    "spark.sql.catalog.aistor.s3.access-key-id": "YOUR-ACCESS-KEY",
    "spark.sql.catalog.aistor.s3.secret-access-key": "YOUR-SECRET-KEY",
    "spark.sql.catalog.aistor.s3.path-style-access": "true",
    "spark.sql.catalog.aistor.io-impl": "org.apache.iceberg.aws.s3.S3FileIO",

    # Iceberg extensions and runtime JARs
    "spark.sql.extensions": "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",
    "spark.sql.defaultCatalog": "aistor",
    "spark.jars.packages": (
        "org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.10.1,"
        "org.apache.iceberg:iceberg-aws-bundle:1.10.1"
    ),
}

Match the iceberg-spark-runtime artifact to your Spark and Scala versions (for example, iceberg-spark-runtime-3.5_2.12 for Spark 3.5 with Scala 2.12). The iceberg-aws-bundle artifact provides the AWS SDK and S3 FileIO that Spark uses for data access.

Trino

Trino connects to the catalog through its Iceberg connector with SigV4 signing. The property keys are the same in both formats; only the SigV4 enablement property differs by Trino version.

SigV4 property differs by Trino version
  • Trino 477 and later: use iceberg.rest-catalog.security=SIGV4.
  • Trino 476 and earlier: use iceberg.rest-catalog.sigv4-enabled=true.

Set the form that matches your Trino version. Do not set both.

Static catalog properties file

Place the following in an iceberg.properties file in the Trino catalog directory (typically etc/catalog/iceberg.properties). This example targets Trino 477 or later.

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=https://aistor.example.net:9000/_iceberg
iceberg.rest-catalog.warehouse=analytics
iceberg.rest-catalog.security=SIGV4
iceberg.rest-catalog.signing-name=s3tables
iceberg.rest-catalog.vended-credentials-enabled=true
iceberg.rest-catalog.view-endpoints-enabled=true
iceberg.unique-table-location=true
s3.region=local
s3.endpoint=https://aistor.example.net:9000
s3.aws-access-key=YOUR-ACCESS-KEY
s3.aws-secret-key=YOUR-SECRET-KEY
s3.path-style-access=true
fs.hadoop.enabled=false
fs.native-s3.enabled=true

For Trino 476 or earlier, replace the SigV4 line:

iceberg.rest-catalog.sigv4-enabled=true

Dynamic catalog creation (SQL)

If your Trino deployment has the CREATE CATALOG SQL syntax enabled, you can create the catalog at runtime. This example targets Trino 477 or later.

CREATE CATALOG aistor USING iceberg
WITH (
    "iceberg.catalog.type" = 'rest',
    "iceberg.rest-catalog.uri" = 'https://aistor.example.net:9000/_iceberg',
    "iceberg.rest-catalog.warehouse" = 'analytics',
    "iceberg.rest-catalog.security" = 'SIGV4',
    "iceberg.rest-catalog.signing-name" = 's3tables',
    "iceberg.rest-catalog.vended-credentials-enabled" = 'true',
    "iceberg.rest-catalog.view-endpoints-enabled" = 'true',
    "iceberg.unique-table-location" = 'true',
    "s3.region" = 'local',
    "s3.endpoint" = 'https://aistor.example.net:9000',
    "s3.aws-access-key" = 'YOUR-ACCESS-KEY',
    "s3.aws-secret-key" = 'YOUR-SECRET-KEY',
    "s3.path-style-access" = 'true',
    "fs.hadoop.enabled" = 'false',
    "fs.native-s3.enabled" = 'true'
);

For Trino 476 or earlier, replace the "iceberg.rest-catalog.security" = 'SIGV4' line with "iceberg.rest-catalog.sigv4-enabled" = 'true'.

TLS and the Java truststore

When MinIO AIStor serves the catalog over HTTPS with a certificate signed by an internal or self-signed certificate authority (CA), the Java runtime that Trino uses must trust that CA. Otherwise, Trino fails catalog connections with a PKIX or “unable to find valid certification path” error.

Import the CA certificate into the truststore that the Trino JVM uses, for example:

keytool -import -alias aistor-ca \
  -file aistor-ca.crt \
  -keystore "$JAVA_HOME/lib/security/cacerts" \
  -storepass changeit

Restart Trino after updating the truststore. Certificates issued by a well-known public CA are already trusted by the default Java truststore and do not require this step.

Starburst and Dremio

Starburst is built on Trino and uses the same Iceberg REST connector and properties shown in the Trino section, including the version-specific SigV4 property and the same Java truststore requirement for internal CAs.

Dremio connects through its Iceberg REST catalog source with SigV4 signing and the s3tables signing name. Use the shared configuration values: the /_iceberg catalog URI, the plain warehouse name, path-style S3 access, and any non-empty region.