Storage Internals

Extent-based storage

Large context blocks are split across multiple NVMe drives to enable parallel I/O. Two dials govern the split:

• storage.block_size — the on-drive extent size. Default 4 MiB. Allocations larger than one extent are striped across drives; smaller allocations live in a single extent. Tune per workload (smaller for many small objects, larger for sustained bulk transfers).
• memory.block_size — the bounce-buffer chunk size used by the RDMA staging pool. Default 2 MiB. This is the granularity of a single RDMA READ / WRITE; multi-extent transfers pipeline through many chunks.

The two are independent — a 4 MiB extent is reached by two 2 MiB bounce-buffer chunks at the default settings.

Versioning and forward compatibility

MemKV releases are date-stamped (RELEASE.<commit-date>). There is no semver line because the on-drive and wire formats are forward-compatible by contract. Every record — superblocks, journal headers, B+Tree pages, the embedded failure registry, and TCP message headers — carries a version field, and a build accepts any version inside the range it supports. Two things fall outside that range and are refused: a version higher than the build's current constant, and a version below the build's floor.

For operators this means:

• Roll-forward is automatic within the supported range. Upgrading the binary keeps the NVMe state usable with no migration step. The new server simply reads the older format.
• A device below the floor must be reformatted. The superblock and B+Tree floor is version 2, because v2 moved the data region to make room for a key region. A server that opens a v1 device refuses it and names the supported range in the error; re-run memkv setup --force on that drive. Its contents are cache state, so reformatting costs repopulation time, not data.
• Don't roll back across format bumps. An older MemKV binary refuses NVMe state written by a newer one rather than risk misinterpreting unknown bytes. Plan upgrades as one-way.
• Mixed-version fleets work in one direction. A newer server accepts requests from older clients (NIXL plugin or admin client); an older server rejects requests from a strictly newer client with UnsupportedVersion. Upgrade servers first, then clients.

The contract is enforced at every header parse site. The wire-protocol header, the device superblock, the embedded failure registry, the journal header, and the B+Tree index header each carry a 1-byte (or 2-byte, for the B+Tree) version field, and each decoder checks it against the range that build supports before reading any other byte.

Capacity reclaim

MemKV holds a cache, so the drives recycle on their own instead of filling up and refusing writes. Once storage passes storage.eviction.high_watermark (70% by default) a background worker releases the least recently used blocks until fill is back under low_watermark (60%), and the freed space goes through the same batched TRIM path as an explicit delete.

This matters most for engines that never delete. SGLang's storage-backend interface has no per-key delete at all — each backend is expected to manage its own retention — so without server-side reclaim a HiCache deployment would fill its drives once and then fail every subsequent write. Set storage.eviction.enabled: false only when the client manages retention and you would rather see writes fail than lose a cached block. See Configuration for the knobs.

Block deletion and TRIM

DELETE operations mark blocks as free in the in-memory index and queue TRIM requests to a background worker. The trim worker batches and coalesces adjacent ranges, then issues NVMe TRIM (BLKDISCARD) commands every 5 seconds or when the batch reaches 1,024 extents. This batched approach avoids synchronous TRIM latency while keeping the SSD controller able to perform garbage collection for sustained write performance.