IN THIS BREAKDOWN · 10 SECTIONS
You can read 14% of this breakdown.
The paper in one minute
A log-structured merge-tree takes every write into memory, logs it, and later writes it to disk in large sorted batches. Random writes become sequential ones. LevelDB and RocksDB are built this way, and so are the storage layers of Bigtable, Cassandra, HBase, TiKV and OceanBase. Yina Lv, Qiao Li and colleagues from Xiamen University, MBZUAI and OceanBase at Ant Group survey what the last five years of research did about the bill that comes with it.
That bill is compaction. Data lands on disk as immutable sorted files (SSTables) in level L0, and background jobs keep merging them into larger levels to remove old versions and deleted keys. Compaction uses the same CPU, disk bandwidth and memory as user requests. When it falls behind, the store throttles or stops writes (a write stall). Compact eagerly and the same data is rewritten many times (write amplification); compact lazily and a read has to look in many files (read amplification).
The survey sorts more than 100 papers from 2020 to 2025 into five questions: how to schedule and shape compaction, how to speed up point and range reads, how to use NVMe, zoned and persistent-memory devices, how the design changes when it is distributed across nodes, and how to keep tenants from hurting each other. It ends with a comparison of single-node, shared-nothing and shared-storage architectures and five open problems. It contains no benchmarks of its own beyond one timing.
Research notes: the problem, approach and results
research questions organizing the survey, each with root causes and fixes (Table 1)
papers from 2020 to 2025 reviewed (Section 1); 154 references in all
for one ELMo-Tune tuning run with DeepSeek V3 under db_bench, the authors' own test (Section 3.1.1)
LSM-trees turn random writes into sequential ones, which is why so many key-value stores use them. The price is compaction: background merging that competes with user traffic for CPU, I/O and memory, causing write stalls, unpredictable latency, write amplification and, if deferred, read amplification. New devices, compute-storage disaggregation and many tenants per node change where those costs land.
Organize the last five years of LSM-tree research around five questions (compaction, point and range reads, emerging storage, single node to distributed, multi-tenant isolation), give each its root causes and families of fixes in one table, and compare single-node, shared-nothing and shared-storage key-value architectures.
A map, not a benchmark: Table 1 lists the fixes per problem with example systems, Table 2 the trade-offs of each architecture, and Section 6 five open directions. The authors' one experiment found LLM-based tuning with ELMo-Tune takes about a hundred seconds per run, too slow for real-time use.
Why it matters to an FDE
You will rarely choose an LSM engine, but you will often run on one. RocksDB sits under Kafka Streams state stores and Flink's RocksDB state backend, CockroachDB uses an LSM engine of its own, and Cassandra, HBase and TiKV are LSM stores. When a customer's pipeline misbehaves under load, the cause is often one of the survey's five problems wearing a different name.
Write latency spikes during a backfill. That is a write stall: the backfill outran compaction, L0 filled up, and the engine slowed or stopped writes to let it catch up.
Deleted data that does not free space. A delete writes a small marker (a tombstone), and the data goes away only when compaction carries that marker to the bottom level. Disk usage can rise after a big delete, and a privacy deletion is not physically complete when the API returns.
Reads that get slower over the day. More files per lookup means more filter checks and more block reads, especially for range scans, which Bloom filters cannot help.
One tenant's batch job slowing everyone. Shared devices and shared compaction threads leak load across tenants unless the design isolates them.
Knowing which problem you are looking at tells you which setting, metric or architecture change to ask about.
Understand the mechanism. Try it on a real problem.
Unlock this breakdown and 199 more papers.
Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.
6 months · One payment · No auto-renewal
- ◇4 diagrams
- </>1 runnable code experiment
- ▦4 results tables
- ?6 interview questions with answers
- 03Key takeaways
The six ideas to keep, the vocabulary, and three misreadings of LSM behavior
Free account - 04How it works
Write and read paths, the four compaction choices, stalls, deletes and the distributed version
Premium - 05The numbers that matter
The survey's few numbers, the Bloom filter formula checked, and the architecture table
Premium - 06Try it yourself
Check the Bloom formula, build leveled and tiered toy stores, and model three stall policies
Premium - 07Where it breaks
What the survey cannot tell you, and five places it contradicts itself or its sources
Premium - 08Using it on a deployment
Diagnosing an LSM store under load: what to collect, a worked stall example and the metrics to watch
Premium - 09Interview angles
Six questions this paper prepares you for, with answer hints
Premium - 10Read next
Where to go from here on this site
Premium