Storage Internals

Extent-based storage

MemKV splits large context blocks across several NVMe drives so it can read and write them in parallel. Two settings control the split:

• storage.block_size is the extent size: the unit of space MemKV allocates on a drive. Default 4 MiB. An allocation larger than one extent is striped across drives; a smaller one fits in a single extent. Use a smaller size for many small objects and a larger size for sustained bulk transfers.
• memory.block_size is the chunk size of the bounce buffers that stage RDMA transfers. Default 2 MiB. Each RDMA READ or WRITE moves one chunk; a transfer that spans several extents flows through many chunks in a pipeline.

The two settings are independent. At the defaults, two 2 MiB chunks fill one 4 MiB extent.

Versioning and forward compatibility

MemKV releases are named by date (RELEASE.<commit-date>), not by semantic version, because the on-drive and wire formats are forward-compatible. Every record carries a version field (1 byte, or 2 bytes in the B+Tree index header): the wire-protocol header, the device superblock, the journal header, the B+Tree index header, and the embedded failure registry. Before it reads any other byte, each decoder checks that version against the range its build supports. A build refuses a version higher than its current one and a version below its floor.

What this means when you upgrade:

• Upgrades within the supported range need no migration. The new server reads the older on-drive format as it is.
• A device below the floor must be reformatted. The floor for the superblock and B+Tree is version 2, because version 2 moved the data region to make room for a key region. A server refuses a version 1 device and names the supported range in the error. Run memkv setup --force on that drive. The drive holds cache data, so reformatting costs only the time to fill the cache again.
• Do not roll back across format changes. An older MemKV binary refuses NVMe state that a newer one wrote, rather than risk misreading it. Treat upgrades as one-way.
• Upgrade servers first, then clients. A newer server accepts requests from older clients (the NIXL plugin or the admin client). If a client uses a higher wire-format version than an older server supports, the server rejects its requests with UnsupportedVersion.

Capacity reclaim

MemKV holds a cache, so it frees space on its own instead of filling the drives and refusing writes. When storage use passes storage.eviction.high_watermark (70% by default), a background worker evicts the least recently used blocks until use drops below low_watermark (60%). The freed space goes through the same batched TRIM path as an explicit delete.

This matters most for engines that never delete. SGLang's storage-backend interface has no per-key delete, because it expects each backend to manage its own retention. Without reclaim on the server, a HiCache deployment would fill its drives once and then fail every later write.

Set storage.eviction.enabled: false only when the client manages retention and you prefer failed writes to losing a cached block. See Configuration for these settings.

Block deletion and TRIM

A DELETE marks blocks as free in the in-memory index and queues a TRIM request to a background worker. The worker merges adjacent ranges and sends NVMe TRIM (BLKDISCARD) commands every 5 seconds, or sooner when the batch reaches 1,024 extents. Batching keeps TRIM latency off the request path, and the TRIM commands let the SSD controller run garbage collection so write performance stays high.