Memory Bucket API

A Memory Bucket is not a plain S3 bucket — it is a bucket that AIStor serves through a dedicated Memory API, the third AIStor API family alongside the Object Store (S3) and Tables APIs. The Memory API is the server-side counterpart to aimem: aimem's verbs (bucket, secret, search) are thin clients over it, and the mount reads and writes ordinary S3 objects into the same bucket.

You never need to call this API directly to use aimem. This page is the wire-level reference for readers who want to see what the mount and the verbs are actually talking to.

The Memory Bucket storage contract

A Memory Bucket is a special S3 bucket that AIStor enforces a storage contract on. AIStor intercepts every write — whether it arrives over the Memory API or a plain S3 PutObject — and makes it honor the contract:

• Hidden. A Memory Bucket does not appear in a regular mc ls alias/ listing (like a Tables warehouse bucket), but it is a real bucket that replication and root can still see.
• Always compressed. Every object is transparently compressed (minlz), regardless of content type.
• Encrypted at rest (with a KMS). With a KMS configured, a Memory Bucket is created with default SSE encryption, so every object is KMS-encrypted at rest and transparently decrypted on read. Without a KMS, objects are compressed but stored unencrypted and the server logs a warning. A KMS is therefore required for the "encrypted" half of the contract to hold. SSE-C is rejected — content is encrypted with the Memory Bucket's own key, not a customer-supplied one.
• Always stamped. Every object carries mandatory memory metadata as ordinary user metadata (x-amz-meta-mem-cortex, x-amz-meta-mem-format), stamped by the server on each write.
• Created fresh or upgraded. A Memory Bucket is normally created via the Memory API. You can also upgrade an existing regular bucket in place (upgradeExisting: true / aimem bucket convert). Objects written before the upgrade are not retro-actively compressed, encrypted, or stamped: the upgrade is recorded on the Memory Bucket itself (upgraded: true) rather than by rewriting old objects. There is no downgrade path.

Root credentials can still GET/PUT into a Memory Bucket over plain S3; the storage contract is enforced regardless, so those writes stay conformant at the storage layer.

The Memory Bucket

A Memory Bucket is the top-level container, analogous to a Tables warehouse. Everything an agent keeps lives on it: the workspace files at the root, the secrets under .secrets/, and the search and structured-memory surfaces described below.

Because a Memory Bucket occupies the Tables warehouse role, it also serves as an Iceberg warehouse. Structured memory — records and events as tables via aimem bio — works on any Memory Bucket, whether created fresh or converted from an existing bucket, with no separate warehouse to provision. aimem bio ships as a separate aimem-bio binary that aimem bio dispatches to (see the CLI reference).

API endpoints

All Memory API requests are served under the /_mem/v1 path prefix and authorized with SigV4. Each maps to a fine-grained memory:* IAM policy action, so a Memory Bucket can be granted to non-owner credentials.

Memory Bucket

CreateCortex takes a JSON body — set upgradeExisting: true to convert an existing bucket, and omit kmsKeyId to take the server default key (SSE-S3) instead of a pinned SSE-KMS key:

{
  "name": "project-memory",
  "kmsKeyId": "optional-kms-key",
  "upgradeExisting": false
}

ListCortexes results are cached (~1s), name-sorted, and support ?search=<substr> plus page-token pagination (?pageToken=, ?pageSize=, default 100 / max 1000). The response carries a nextPageToken when more results remain.

Secrets (the Vault form)

Secret values live under the Memory Bucket's reserved .secrets/ prefix, full-file KMS envelope-encrypted — a plain S3 GET returns ciphertext, while an authed Memory API GetSecret returns the server-decrypted plaintext.

Search over a Memory Bucket

Search is a first-class Memory API operation: server-side RE2 regex matching over a Memory Bucket's objects at rest, served under the Memory Bucket path. The server decrypts and decompresses each object in place, skips blocks it can rule out, and streams back only the hits. This is how an agent searches across its long-term memory without downloading the Memory Bucket.

How aimem verbs map to the API

The workspace itself needs no Memory API calls at all: mounted files are ordinary S3 objects written to plain, path-shaped keys at the root of the Memory Bucket bucket.

Data layout

Inside the Memory Bucket bucket, workspace files sit at the root and three prefixes are reserved — managed only through the Memory API and rejected for direct S3 writes:

project-memory/
├── <workspace files…>   workspace: plain path-shaped keys at the root
├── .cortex/             reserved: Memory Bucket write-path policy (replicated)
├── .secrets/<name>      reserved: Secrets API — envelope ciphertext
└── .mem/                reserved: future Memory API use

Reserving .mem/ up front guarantees a workspace file can never collide with future Memory API use of that prefix.

What the Memory API delivers

• Memory Bucket. The Memory Bucket: hidden bucket, transparent compress + encrypt, mandatory metadata stamping, and Memory Bucket CRUD.
• Secrets. The Vault form — envelope-encrypted secret values under .secrets/.
• Search. Server-side RE2 regex search over the objects at rest — decrypted and decompressed server-side, matched in place, hits streamed back.