AIStor Memory
Concepts

Architecture

How AIStor Memory presents an agent's memory as files, how reads and writes reach the Memory Bucket, what the local cache holds, and how the agent skill is installed.

How the memory is presented as files

AIStor Memory data flow — agent in a sandbox writes through the mount to the aimem binary, which stages writes on NVMe and serves cached reads from local disk via FUSE passthrough; both rails reach a MinIO AIStor Memory Bucket over S3.

AIStor Memory presents an agent's workspace and long-term memory as ordinary files. The aimem binary mounts a filesystem at a directory you choose and turns the agent's file operations into requests to an AIStor Memory Bucket. It refuses to mount a plain bucket.

  • Reads come from a local cache when the file is there, and from the bucket otherwise.
  • Writes go to a local staging file first. The file uploads to the bucket as one complete object when the agent closes it.

The bucket stores the original file bytes, not a summary made by a model, in storage you run, with your own keys. There is no model on the write path, no separate database or metadata cluster, and no process to run besides aimem.

Secrets are the exception: they are never written to the mount. The launcher passes them to the agent as environment variables. See Vault.

What ships in the binary

CapabilityWhat it gives the agent
Workspace stagingEach file opened for writing is backed by a local copy-on-write file on NVMe, so git, editors, sqlite, and in-place edits work. See Write path.
Atomic uploadOn close() or fsync(), the staged file uploads in one request or as a chunked upload, depending on size. The bucket sees either the whole new file or none of it.
Local read cache and warmupThe --local store is also a read cache. On mount, AIStor Memory copies the agent's memory files (CLAUDE.md, .claude/, AGENTS.md, .cursor/, MEMORY.md, …) into it in the background, so first reads usually come from local NVMe. A read that arrives before warmup finishes goes to S3.
Read-aheadSequential reads fetch more data ahead of time; random reads fetch less.
Agent skill deliveryThe aimem Agent Skill reaches the agent with no setup. See Skill delivery.
Multi-endpoint S3 transportRound-robin across endpoint_url: [http://aistor{1...4}:9000], with optional health checks and per-request timeouts. See Configuration.
Endpoint-provisioned secretsaimem secret provision runs on the launcher and passes the granted secrets to the agent as environment variables. It reads them from the Memory Bucket (--bucket, the main path) or from an HTTP secret store such as HashiCorp Vault or OpenBao (--store-endpoint / --store-token). aimem secret put/get/list/delete <bucket> <name> manage secrets in the Memory Bucket. aimem stores no secret bytes; the Memory Bucket keeps them encrypted on the server.
Memory Bucket lifecycleaimem create / convert / list / get / delete / credentials run on the launcher. They create a Memory Bucket (or convert a plain bucket with convert or --upgrade-existing), show its properties (encryption, compression, KMS key, memory format v1, upgraded flag), and issue STS credentials scoped to one Memory Bucket.
Server-side searchaimem search <bucket> -e <regex> runs RE2 pattern matching on the server over the stored objects and returns only the matches. The agent does not download the workspace. When you pass several -e patterns, all must match. --ignore-case, --prefix, and --ext narrow the search.
Required staging directoryYou must set the local staging directory with the local: key in --config, the --local <DIR> flag, or the AIMEM_LOCAL environment variable. The flag and environment variable take precedence over the config key. There is no default.

Mount preconditions

AIStor Memory mounts only a Memory Bucket. Before the filesystem comes up, it checks four things. If any check fails, the mount stops with an error:

  1. An endpoint. Set the endpoint_url: config key or the --endpoint-url flag.
  2. Signing credentials. Set AIMEM_ACCESS_KEY and AIMEM_SECRET_KEY, and optionally AIMEM_SESSION_TOKEN. Anonymous mounts and default credential chains are refused.
  3. A Memory Bucket. AIStor Memory calls the Memory API GetCortex on the bucket. A plain bucket is rejected with a hint to run aimem create.
  4. Access to the Memory API. The Memory API is an AIStor Enterprise feature. If access is denied, the error reads "the credentials lack memory:GetCortex, or the server is not on a paid tier."

A backend that is not AIStor fails on the first rename().

A mount needs the required version key and a staging directory. This example sets the staging directory and endpoint in the config file and takes the credentials from the environment:

# /etc/aimem/mount.yaml
version: "1.0"
local: /var/cache/aimem
endpoint_url:
  - https://aistor.example.com:9000
export AIMEM_ACCESS_KEY=… AIMEM_SECRET_KEY=…
aimem <bucket> /workspace --config /etc/aimem/mount.yaml

--local /var/cache/aimem and AIMEM_LOCAL=/var/cache/aimem set the same staging directory. The mount reads the endpoint only from --endpoint-url or endpoint_url:. It does not read AIMEM_ENDPOINT_URL, which is for the management commands.

Read path

read(2)
  └─→ FUSE
        ├─→ --local cache hit → FUSE passthrough  (kernel reads the backing fd directly)
        └─→ miss → prefetcher  (advisory read-ahead, sequential growth)
                     └─→ S3 GetObject → written through into --local

The local store (local:, --local, or AIMEM_LOCAL) is both the write staging area and the read cache. You size and manage one disk store, not two.

  • Cache hit. When a file opened for reading is already in --local (from warmup or an earlier read), the kernel reads the local copy directly through FUSE passthrough.
  • Cache miss. AIStor Memory streams the object from S3 with read-ahead and saves the bytes in --local for the next read.
  • Warmup. On mount, AIStor Memory loads the agent's startup files (CLAUDE.md, .claude/, AGENTS.md, …) into --local in the background. The agent config key selects which files. Warmup never delays the mount.

Reading a file while it is being written. A file descriptor opened for reading follows the file as it changes. While another descriptor writes the file, reads return the writer's staged bytes, including a truncate at open. After the writer commits, the next read returns the new object, even if the reader had buffered the old one. Three kinds of descriptor keep the bytes they were opened on:

  • A descriptor opened while a writer is active reads that writer's staged file. It does not follow a later writer.
  • A descriptor whose file was deleted or renamed over keeps reading the file it opened, as POSIX requires.
  • A descriptor that uses FUSE passthrough reads the cached copy. Passthrough is used only for read-only opens of files already in --local.

Write path

Every file opened for writing goes to the staging directory, as a local copy-on-write file on NVMe. The staged file uploads to S3 on close() or fsync(). The upload is atomic: the bucket never holds a half-written file.

Because the agent writes to a local file, its tools work unchanged:

  • Random writes work, because the staged file is a normal local file.
  • O_RDWR reads and writes the staged file through the same handle.
  • O_TRUNC truncates the staged file, and the upload commits the truncated result.
  • mmap and copy_file_range return ENOSYS. The kernel falls back to plain read and write on the staged file, so callers still work.
  • vim, git, and other tools that edit in place see a regular filesystem.

The staging directory needs enough free space for all files the agent has open for writing at once. Use NVMe.

No metadata cluster

AIStor Memory assumes one writer per mount, so it needs no separate metadata database. File and directory listings come from the bucket. The kernel caches this metadata for the time set by metadata_ttl. There is no extra process to deploy, monitor, scale, or fail over.

Skill delivery

AIStor Memory ships an Agent Skill named aimem that teaches the agent how to use the mount. It is delivered in three tiers. The first two happen on every mount, with no flags. For details, see SKILLS.

  1. Read-time preamble. When the agent reads an instruction file (CLAUDE.md, AGENTS.md, .cursorrules) from the mount, AIStor Memory adds a short orientation preamble to the start. The preamble exists only in memory and is never written to the bucket. It is off when the mount root is a git repository. To turn it off, set AIMEM_PREAMBLE_OVERLAY=0.
  2. Home directory install. AIStor Memory installs the skill into the agent's home skill directories: $HOME/.agents/skills/aimem/ for Codex CLI, Cursor, Gemini CLI, and Copilot, and $HOME/.claude/skills/aimem/ for Claude Code (honoring CLAUDE_CONFIG_DIR). The install needs no network and never overwrites existing files. It is skipped when $HOME is inside the mount. To turn it off, set AIMEM_INSTALL_SKILL=0.
  3. Public repository. An agent that runs outside the sandbox installs the same skill on the host with npx skills add minio/skills/aimem from minio/skills.