GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo
ICLR 2026

AtlasKV: Augmenting LLMs with Billion-Scale Knowledge Graphs in 20GB VRAM

AtlasKV puts knowledge-graph facts inside a model's attention instead of its prompt. Its 1B-triple, 20 GB headline is a memory result; accuracy was measured up to 10,000 triples.

Breakdown by FDE Interviews · 14 min read · Updated

About the original · 2025
AUTHORS
Huang, Tsang, Bai, Peng, Zhang, Song
FROM
The Hong Kong University of Science and Technology, Theory Lab, Huawei
PUBLISHED
Oct 2025 · ICLR 2026
CITED BY
7Semantic Scholar · Nov 2026
IN THIS BREAKDOWN · 10 SECTIONS

You can read 17% of this breakdown.

01

The paper in one minute

Retrieval-augmented generation finds relevant text and pastes it into the prompt. That works, but every query pays for a search, and a long retrieved context makes the model slower. AtlasKV, an ICLR 2026 paper from HKUST and Huawei, takes a different route that it builds on from an earlier method, KBLaM (Wang and colleagues, 2024): encode each fact once as a key and a value vector, and let the model's attention layers look facts up directly, with no retriever and no long prompt.

The paper adds two things. KG2KV turns every knowledge-graph triple, such as (John founded StockLemon.com, because, John made profits), into training examples by masking one end and rewriting the relation as a noun: the key becomes "the cause of John founded StockLemon.com" and the value is the masked event. That gives far more varied training questions than KBLaM's synthetic data. HiKVP groups the keys into a three-layer tree and, at every attention layer, keeps only the best 128 top-level keys, then 64 middle keys, then 16 facts. The model sees 16 facts per layer whatever the size of the graph.

The headline in the title is about memory: under 20 GB of GPU memory with a graph of one billion triples, where KBLaM needs over 40 GB at 100,000. Accuracy was measured on graphs up to 10,000 triples, and pruning lowers it. Both facts are in the paper; the social-media version of it kept only the first. The paper was first posted in October 2025.

THE IDEA, VISUALLY
KG2KV: A TRIPLE BECOMES Q, K, V (John founded StockLemon.com, because, John made profits ...) mask the tail; rewrite the relation as a noun Q What is the cause of John founded StockLemon.com? K the cause of John founded StockLemon.com V John has made profits every year ... Encoded by all-MiniLM-L6-v2; new projection heads are trained. HIKVP: PRUNE A POOLED TREE AT EVERY LAYER root keys pooled; on GPU keep top 128 middle keys moved in from CPU keep top 64 leaf keys moved in from CPU keep top 16 16 values join the model's own attention, weighted against the context by their logits Cluster size S = ceil(M^(1/3)): about 193,000 keys scored per layer at M = 1 billion (our arithmetic, Part 1). WHAT WAS MEASURED, BY KG SIZE GPU memory (Figure 4) grounding accuracy (Table 3) not measured 10^1 10^3 10^5 10^7 10^9 Under 20 GB at 1 billion triples is a memory claim. TOP-1 GROUNDING, ATLAS-CC-QKV (TABLE 3) 10^2 triples 96.4 89.1 10^3 triples 83.6 61.8 10^4 triples 61.8 40.0 blue = with pruning (the 1B-triple setting), gray = without AVERAGE INFERENCE TIME, SECONDS (TABLE 11) 10^1 4.10 vs 1.00 10^2 4.02 vs 1.98 10^3 4.44 vs 2.11 10^4 6.21 vs 2.09 red = with pruning, gray = without: memory is bought with latency
Left: KG2KV turns each triple into query, key and value strings, and HiKVP keeps only 128, 64 and 16 keys at three layers of a pooled tree before the surviving values join attention. Right: memory was measured to one billion triples and accuracy to ten thousand. Pruning cost about 22 points of Top-1 accuracy at ten thousand triples on ATLAS-CC-QKV and made inference two to four times slower.
Research notes: the problem, approach and results
<20 GB

GPU memory at 1B triples with HiKVP (abstract, Figure 4)

10^4

largest knowledge graph with an accuracy number (Table 3)

40.0%

Top-1 grounding at 10^4 triples with pruning, ATLAS-CC-QKV (Table 3)

THE PROBLEM

RAG over a very large knowledge base pays for search and for long prompts; earlier in-attention methods such as KBLaM grow linearly with the knowledge base and generalize poorly.

THE IDEA

Rewrite each knowledge-graph triple as a query-key-value example, train small projection heads once, and at inference prune a three-layer tree of keys so only a few facts enter attention.

THE RESULT

Under 20 GB of GPU memory at one billion triples, and better grounding than KBLaM on out-of-distribution sets up to 10,000 triples. Pruning trades accuracy and latency for that memory.

02

Why it matters to an FDE

This paper circulated on LinkedIn in November 2026 as proof that RAG is obsolete, a year after it was first posted. Expect a customer to forward it and ask whether their retrieval pipeline should be replaced. The useful answer separates three claims: memory, accuracy and latency. AtlasKV makes a convincing memory claim. Its accuracy evidence covers graphs a million times smaller than its headline, using a grounding measure read from attention scores in one layer. Its own appendix shows pruning made inference two to four times slower in that setup.

The idea is still worth understanding. Putting facts into attention as keys and values, with a trained lookup, is a real alternative to pasting retrieved text, and it removes the prompt-length cost and the "lost in the middle" effect for those facts. It needs a fine-tuned model with extra projection heads, a knowledge base already in triple form, and an inference stack that can move keys between CPU and GPU memory. Few customer environments have all three, and hosted model APIs offer none of them.

So the practical position for most deployments today is to keep retrieval, read AtlasKV as research worth tracking, and know which experiment would change that. For the general trade-off, the RAG vs long context breakdown and the retrieval-augmented generation concept are the places to start.

THE FULL BREAKDOWN

Understand the mechanism. Try it on a real problem.

Unlock this breakdown and 297 more papers.

Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.

6 months · One payment · No auto-renewal

Get Premium · ₹2,000 / $25
  • ◇1 architecture diagram
  • </>2 runnable code blocks
  • ▦3 results tables
  1. 03Key takeaways

    Six points to carry into a conversation about replacing retrieval with in-attention knowledge

    Free account
  2. 04How it works

    Follow a triple from text to attention, and a query through the three-layer pruning tree

    Premium
  3. 05The numbers that matter

    Memory, grounding accuracy, answer quality and latency, each with its scope

    Premium
  4. 06Try it yourself

    Recompute HiKVP's work per query, then watch fixed beams lose the right fact as a store grows

    Premium
  5. 07Where it breaks

    Scale beyond the tested graphs, latency, multi-hop questions, model access and changing facts

    Premium
  6. 08Using it on a deployment

    The prerequisites, a worked evaluation plan, and the numbers that would justify switching

    Premium
  7. 09Interview angles

    Questions about in-attention knowledge, the RAG comparison and reading a scaling claim

    Premium
Back to the paper library →