AIStor Memory

Architecture

How the memory an agent reads and writes is presented as files — the read and write data path, the local cache, memory warmup, and the agent skill AIStor Memory installs. Your storage, your keys; real files, with no model on the write path and no separate database to run.

How the memory is presented as files

AIStor Memory data flow — agent in a sandbox writes through the mount to the aimem binary, which stages writes on NVMe and serves cached reads from local disk via FUSE passthrough; both rails reach a MinIO AIStor Memory Bucket (cortex) over S3.

The memory an agent reads and writes — its workspace and its long-term memory — is presented as ordinary files. (The third form, the vault, is different by design: secrets are provisioned launcher-side as environment variables and never written to the mount — see the secrets model.) AIStor Memory is a single static Rust binary that mounts a filesystem under any directory you pass it and turns the agent's file operations into calls against a MinIO AIStor Memory Bucket (a cortex) — a plain bucket is refused at mount. Reads stream through a prefetcher with read-ahead; writes are staged on local NVMe and uploaded atomically on close().

What lands in the cortex is the bytes themselves — the original file, not a summary rebuilt by a model — in storage you already run, with your own keys. There is no model on the write path, no separate database to run, and nothing to reconstruct on read.

No external services. No metadata cluster. No daemons besides aimem itself. The rest of this page explains how the mount reaches that: what ships in the binary, the mount preconditions, and the read and write data paths.

What ships in the binary

CapabilityWhat it gives the agent
Workspace stagingEvery open-for-write is backed by a local copy-on-write file on NVMe, so git, editors, sqlite, and in-place edits all work against it (the syscall-level details are below).
Atomic uploadThe atomic upload (all-or-nothing — the bucket only ever sees a complete file) is committed on close() / fsync(): the staged file uploads to S3 in a single shot or chunked upload depending on size. Either the bucket sees the whole new file or none.
Local read cache + warmupThe --local store doubles as a read cache: vendor-specific memory paths (CLAUDE.md, .claude/, AGENTS.md, .cursor/, MEMORY.md, …) are warmed into it on mount (in the background) so first-touch reads are served from local NVMe via FUSE passthrough instead of S3.
Read-ahead prefetcherSequential-access reads grow a read-ahead window automatically; random-access reads shrink it. Stays out of the way otherwise.
Agent skill deliveryThe aimem Agent Skill reaches the agent automatically — a read-time preamble prepended to instruction files read from the mount, plus a create-only install into $HOME skill directories — so the model arrives primed on navigation, workspace, memory, metadata, annotations, and search.
Multi-endpoint S3 transportRound-robin across endpoint_url: [http://aistor{1...4}:9000] with optional health-checking and per-request timeouts.
Endpoint-provisioned secretsaimem secret provision resolves the granted secrets — from the cortex over the Memory-API secrets endpoint (--cortex, the primary path) or, as a bridge, from a generic HTTP store (--store-endpoint / --store-token, e.g. HashiCorp Vault/OpenBao or a cloud manager) when secrets already live there — running launcher-side, and injects them into the agent as environment variables at launch. aimem secret put/get/list/delete <cortex> <name> manage secrets stored in the cortex over the Memory API (encrypted server-side). aimem stores no secret bytes itself (pass-through client); the cortex persists them encrypted server-side.
Cortex lifecycleaimem cortex create / convert / list / get / delete / credentials manage AIStor Memory Buckets launcher-side over the Memory API: create a cortex (or convert / --upgrade-existing a plain bucket into one), inspect its properties (encryption, compression, KMS key, memory format v1, upgraded flag), and mint STS credentials scoped to a single cortex.
Server-side searchaimem search <cortex> -e <regex> runs RE2 pattern matching entirely server-side over objects at rest — the server decrypts and decompresses server-side, matches in place, and streams matches back; the agent never downloads the workspace. Multiple -e patterns are ANDed (all must match); --ignore-case, --prefix, and --ext narrow the scan.
Required staging dirThe local staging directory is required and may be supplied three ways — the local: key in --config, the --local <DIR> flag, or the AIMEM_LOCAL env var; at least one is required, and CLI/env win when more than one is set. There is no auto-detection or default.

Mount preconditions

AIStor Memory mounts only a cortex — there is no plain-bucket mode. Before the filesystem comes up, the production client verifies four things, each fail-closed with an explicit error:

  1. An explicit endpoint — the endpoint_url: config key (or the --endpoint-url flag). A cortex mount cannot be verified without one.
  2. Real signing credentials — AIMEM_ACCESS_KEY / AIMEM_SECRET_KEY (plus optional AIMEM_SESSION_TOKEN). Anonymous or default-chain mounts are refused because they cannot be verified.
  3. The target is a cortex, not a plain bucket — AIStor Memory probes the Memory API GetCortex; a regular bucket is rejected with a hint to run aimem cortex create.
  4. Paid tier / access — the Memory API is an AIStor Enterprise capability. Access-denied surfaces as "the credentials lack memory:GetCortex, or the server is not on a paid tier."

AIStor Memory does not probe for AIStor-vs-vanilla-S3 at mount (the Server header is unreliable behind proxies); a non-AIStor backend instead surfaces a rename error on the first rename().

A mount supplies the required staging dir through the config's mandatory version and local: keys; the endpoint can live there too, and the signing credentials come from the environment:

# /etc/aimem/mount.yaml
version: "1.0"
local: /var/cache/aimem
endpoint_url:
  - https://aistor.example.com:9000
export AIMEM_ACCESS_KEY=… AIMEM_SECRET_KEY=…
aimem <cortex> /workspace --config /etc/aimem/mount.yaml

--local /var/cache/aimem and AIMEM_LOCAL=/var/cache/aimem are the flag and env equivalents for the staging dir; supply at least one of the three (CLI/env win when more than one is set). The endpoint itself is supplied by the --endpoint-url flag or the endpoint_url: config key (the mount does not read AIMEM_ENDPOINT_URL — that variable is for the management verbs).

Read path

read(2)
  └─→ FUSE
        ├─→ --local cache hit → FUSE passthrough  (kernel reads the backing fd directly)
        └─→ miss → prefetcher  (advisory read-ahead, sequential growth)
                     └─→ S3 GetObject → written through into --local

The local store (the canonical local: config key; also --local / AIMEM_LOCAL) is both the write staging area and the read cache. On a read-mode open, if the object is already in --local (warmed on mount, or written through by an earlier read), AIStor Memory hands the kernel the backing file descriptor via FUSE passthrough — subsequent reads bypass the userspace data path entirely. On a miss the prefetcher streams the object from S3, growing its read-ahead window on sequential access and shrinking it on random access, and the bytes land in --local for next time. The agent-memory warmup pass (driven by the agent YAML key) pre-populates --local on mount in the background — never blocking the mount — so the bootstrap-read set (CLAUDE.md, .claude/, AGENTS.md, …) is already local on first touch.

Why this shape. A generic mount either treats every read as cold against the backend, or caches everything in a separate in-memory or SSD tier. AIStor Memory keeps a single local store: --local is both the write-through staging area and the read cache, with warmup priming exactly the small agent-memory set an agent reads on startup and the prefetcher handling everything else. One disk store, no second cache tier to size or operate.

Write path

Workspace mode is always on. Every open-for-write hits the staging directory — a local copy-on-write file on NVMe — and the staged file uploads to S3 on close() / fsync().

This is what lets the agent's tools work unchanged:

  • Random writes — the file is mutable on the local filesystem.
  • O_RDWR — same handle reads and writes against the staged file.
  • O_TRUNC — local truncation, plus the upload commits the truncated result.
  • mmap, copy_file_range — return ENOSYS; the kernel falls back to plain read/write against the staged file, so callers still work.
  • In-place edits with vim, git, etc. — they see a regular filesystem and never have to learn S3 semantics.

The trade-off: the staging directory needs enough free space for the agent's open-write working set. NVMe is strongly recommended.

Why this shape. Bolting a network filesystem straight onto object storage — the legacy gateway approach — either rejects random writes outright or relies on a best-effort upload: a race during close() can leave the bucket with a half-written object. AIStor Memory's atomic upload guarantees the bucket only ever sees a complete file — local staging for a normal-filesystem experience, fenced upload on close().

No metadata cluster

One way to deliver POSIX-correctness on object storage is to externalise the filesystem metadata into a separate database (Redis, Postgres, or a dedicated metadata tier). That's viable for a mount with thousands of concurrent writers, but it adds a whole second system to operate — and agent sandboxes are exactly the wrong place to add operational surface.

AIStor Memory's single-writer-per-mount model gets away without a metadata cluster entirely. Inodes and directory listings come from S3 itself (via ListObjectsV2 with delimiter); FUSE caches the metadata for the duration set by metadata_ttl. There's no separate process to deploy, monitor, scale, or fail over.

Skill delivery

AIStor Memory ships a first-class Agent Skill named aimem and delivers it in three tiers, the first two automatic on every mount — no flags. See the SKILLS section of the aimem reference for the canonical description.

  1. Read-time preamble overlay — instruction files (CLAUDE.md, AGENTS.md, .cursorrules) read from the mount are served with a short orientation preamble prepended. The injection is in memory only, never written back to the bucket; it is disabled automatically when the mount root is a git repository, and can be turned off with AIMEM_PREAMBLE_OVERLAY=0.
  2. Home-dir install — AIStor Memory installs the aimem skill into the agent's home skill directories ($HOME/.agents/skills/aimem/ for Codex CLI, Cursor, Gemini CLI, Copilot; $HOME/.claude/skills/aimem/ for Claude Code, honoring CLAUDE_CONFIG_DIR). The write is offline and create-only, skipped when $HOME resolves under the mount, and opted out with AIMEM_INSTALL_SKILL=0.
  3. Public repo — agents whose reasoning runs outside the sandbox install the same skill host-side with npx skills add minio/skills/aimem from minio/skills.

Because the memory is a mount of real files, there is something concrete for the agent to read and an in-sandbox agent to orient: AIStor Memory meets the agent where it already looks — the bytes it reads on startup and the skill directories it already loads. For an agent running in the sandbox the skill installs automatically, with no separate integration step (unless $HOME resolves under the mount, where the install is skipped); an agent whose reasoning runs outside the sandbox adds it host-side with npx skills add minio/skills/aimem, as above.

Operations

Running the memory your agents keep in production — per-sandbox-provider mount recipes, fstab and daemonization, logging, OpenTelemetry metrics, seeding the Vault, and troubleshooting a cortex mount.

Advanced

The machinery under the mount — the AIStor Memory Bucket API that a cortex speaks, how aimem's verbs map onto it, and the cortex storage contract. For readers who want the wire-level view; nothing here is required to use aimem.

On this page

How the memory is presented as filesWhat ships in the binaryMount preconditionsRead pathWrite pathNo metadata clusterSkill deliveryWhere to read next