AIStor Tables

This page provides an example that shows how to create an AIStor Tables warehouse and perform basic table operations with the AIStor Client and PyIceberg, a lightweight Python API for interacting with Iceberg Tables. You use the client to create the needed resources and Python code to insert and query data using the AIStor Tables API.

You can also call the AIStor Tables API directly, instead of working with the AIStor client commands. See the AIStor Tables API Reference for more information.

For a high-level introduction and quickstart, see also the AIStor Tables administration documentation.

Prerequisites

The AIStor Client command-line tool configured with an alias for your cluster
Python 3.9 or later.
PyIceberg, PyArrow, and pandas.

You can install the required Python packages with the following command:

pip install pyiceberg pyarrow pandas

You must perform the following steps in the order specified.

Create the warehouse

A warehouse is the top-level container in AIStor Tables that holds namespaces and tables. Creating a warehouse also creates a corresponding bucket to store Iceberg table data and metadata. While you can inspect some details of this bucket with standard AIStor Client commands, you can only manage its contents with the AIStor Tables API or mc table commands. This protects the integrity of the underlying objects that contain your table data.

Create a warehouse named mywarehouse in the AIStor cluster myaistor:

mc table warehouse create myaistor mywarehouse

Create the namespace

Namespaces organize tables within a warehouse. You can create as many namespaces as you need to manage multiple tables within a single warehouse.

Create a namespace named mynamespace inside the warehouse:

mc table namespace create myaistor mywarehouse mynamespace

Create the table

Create a table with a schema defining the columns and their types, taking care to match not only the types of the expected fields but also whether or not a particular field must contain a value or can be empty. Schemas follow the Iceberg schema specification.

Create a table named mytable with five columns:

Example

mc table create myaistor mywarehouse mynamespace mytable                     \
  --schema '{"type":"struct","fields":[                                      \
            {"id":1,"name":"id","type":"long","required":true},              \
            {"id":2,"name":"name","type":"string","required":false},         \
            {"id":3,"name":"category","type":"string","required":false},     \
            {"id":4,"name":"price","type":"double","required":false},        \
            {"id":5,"name":"in_stock","type":"boolean","required":false}]}'

Copyable example

mc table create myaistor mywarehouse mynamespace mytable \
  --schema '{"type":"struct","fields":[{"id":1,"name":"id","type":"long","required":true},{"id":2,"name":"name","type":"string","required":false},{"id":3,"name":"category","type":"string","required":false},{"id":4,"name":"price","type":"double","required":false},{"id":5,"name":"in_stock","type":"boolean","required":false}]}'

The schema defines these columns:

Column	Type	Required	Description
`id`	long	Yes	Unique identifier
`name`	string	No	Product name
`category`	string	No	Product category
`price`	double	No	Product price
`in_stock`	boolean	No	Whether item is in stock

You can verify the table was created with mc table list:

mc table list myaistor mywarehouse mynamespace

View table metadata with mc table show:

$ mc table show myaistor mywarehouse mynamespace mytable

The output resembles the following:

Table: mytable
Namespace: mynamespace
Warehouse: mywarehouse
Location: s3://mywarehouse/5e00905e-be0f-4799-8501-9dda7d48b695
Format Version: 2
Table UUID: 5e00905e-be0f-4799-8501-9dda7d48b695
Schema:
  - id: long (required)
  - name: string
  - category: string
  - price: double
  - in_stock: boolean

Run the sample application

Now that the table exists, you can work with it using the AIStor Tables API.

Download the hello-aistor-tables.py Python example and save it on your local system.

Run the example with the following command:

python hello-aistor-tables.py

The script performs these operations:

Connects to the AIStor Tables catalog using SigV4 authentication.
Loads the existing table created with mc table create.
Displays the table schema and metadata.
Appends three sample product records to the table.
Queries the data using different filter expressions.

Expected output

The script produces output similar to:

============================================================
AIStor Tables Basic Example
============================================================
Connected to catalog at http://localhost:9000/_iceberg
Loaded table: mynamespace.mytable

--- Table Information ---
Name: ('mynamespace', 'mytable')
Location: s3://mywarehouse/f07c5466-6929-46cd-b0f5-57335bd2fa09

Schema:
  id: long (required)
  name: string (nullable)
  category: string (nullable)
  price: double (nullable)
  in_stock: boolean (nullable)

============================================================
Appending Data
============================================================
Appended 3 rows to table
Loaded table: mynamespace.mytable

============================================================
Query Examples
============================================================

--- Query: All Rows ---
Found 3 total rows
    id      name     category  price  in_stock
0  101  Widget A  Electronics  29.99      True
1  102  Widget B  Electronics  49.99      True
2  103  Widget C     Hardware  19.99     False

--- Query: Filter 'price > 30' ---
Found 1 matching rows
    id      name     category  price  in_stock
0  102  Widget B  Electronics  49.99      True

--- Query: Filter 'in_stock = true' ---
Found 2 matching rows
    id      name     category  price  in_stock
0  101  Widget A  Electronics  29.99      True
1  102  Widget B  Electronics  49.99      True

--- Query: Filter 'category = 'Electronics'' ---
Found 2 matching rows
    id      name     category  price  in_stock
0  101  Widget A  Electronics  29.99      True
1  102  Widget B  Electronics  49.99      True

--- Query: Columns ('name', 'price') ---
Found 3 rows
       name  price
0  Widget A  29.99
1  Widget B  49.99
2  Widget C  19.99

--- Query: Columns ('name', 'price', 'in_stock') ---
    Filter: category = 'Electronics'
Found 2 rows
       name  price  in_stock
0  Widget A  29.99      True
1  Widget B  49.99      True

--- Query: Limit 5 rows ---
Retrieved 3 rows
    id      name     category  price  in_stock
0  101  Widget A  Electronics  29.99      True
1  102  Widget B  Electronics  49.99      True
2  103  Widget C     Hardware  19.99     False

============================================================
Snapshot After Append
============================================================

--- Snapshot Information ---
Snapshot ID: 3254159768753409734
Timestamp: 1769554450753
Operation: Operation.APPEND

============================================================
Example Complete
============================================================

Clean up

To delete the resources created in this tutorial, remove them in the reverse order they were created. Once you have removed all the AIStor Tables resources for a warehouse, AIStor converts the underlying bucket to a standard AIStor Object Store bucket. You can then inspect or manage any remaining objects in the bucket with standard AIStor Client commands such as mc ls.

# Remove the table
mc table remove myaistor mywarehouse mynamespace mytable

# Optional: also permanently delete table files when removing the table
mc table remove myaistor mywarehouse mynamespace mytable --purge

# Remove the namespace (must not contain tables)
mc table namespace remove myaistor mywarehouse mynamespace

# Remove the warehouse (must not contain namespaces)
mc table warehouse remove myaistor mywarehouse

# Optional: remove the warehouse bucket (must be empty and not linked to a table)
mc rb myaistor/mywarehouse