MemOS: A Memory OS for AI System

A language model "remembers" in three ways. Parameter memory is what training put in the weights. Activation memory is the attention key-value cache built while reading a context. Plaintext memory is text supplied from outside: documents, notes, retrieved passages. In most applications these are handled separately, by training, by the serving engine and by a RAG pipeline, with no common way to track where a memory came from, who may use it, when it should expire, or when it should move from one form to another.

Zhiyu Li, Feiyu Xiong and a large team at MemTensor (Shanghai) and several Chinese universities propose MemOS, a memory operating system that manages all three. Its basic unit is the MemCube: a payload (text, a KV cache, or a parameter module such as a LoRA adapter) plus metadata in three groups. Descriptive identifiers: timestamp, origin (extracted by inference, entered by a user, retrieved, or fine-tuned) and semantic type. Governance attributes: access scope, lifespan, priority, and compliance fields such as sensitivity tags and watermarks. Usage indicators: access frequency and recency, which tell the system whether a memory is hot or cold.

MemOS manages MemCubes through three layers. The interface layer parses requests into structured memory calls (MemReader) and exposes a Memory API. The operation layer organizes and retrieves memories (MemOperator), decides what to load and in which form (MemScheduler), and tracks states over time (MemLifecycle). The infrastructure layer enforces access control, retention and audit (MemGovernance), stores memories in repositories (MemVault), imports and exports them (MemLoader, MemDumper), and shares them between agents and organizations (MemStore). Memories can also change form: frequently used text can become a KV cache for faster decoding, and stable knowledge can be distilled into parameters.

The evaluation compares the MemOS system with six memory products (MIRIX, Mem0, Zep, Memobase, MemU and Supermemory), all on GPT-4o-mini, across LoCoMo, LongMemEval, PrefEval and PersonaMem. MemOS had the best overall score on each, and the lowest latency under load in an API stress test. A separate experiment shows that injecting stored memory as a KV cache cuts time to first token sharply compared with pasting it into the prompt.

Enterprise customers ask the same questions about any memory feature: whose memory is this, who can see it, how long is it kept, can it be deleted, and why did the assistant use it? MemOS is useful mainly as a checklist for those questions.

Metadata is the product. Origin, access scope, lifespan, sensitivity and usage per memory item are what make memory governable. Design them in from the start, whatever system you use.

KV-cache memory is a latency tool, not free storage. Reusing a precomputed cache for a stable context, such as a policy manual or a customer profile, can cut time to first token by an order of magnitude. Each cached token costs GPU memory, about 144 KiB on an 8B-class model, so only a few hot contexts belong there.

Vendor benchmarks need a second look. The authors built MemOS and also configured and ran their competitors. They chose each method's configuration "based on its best validation performance", and stress-tested rivals' hosted APIs. Re-run the comparison on your customer's data before choosing.

Parameter memory is the slow tier. Turning knowledge into adapters is the hardest transition to govern: it cannot be deleted item by item. Keep personal data out of it.