How Fast Does KV Cache Offloading Wear Out SSDs?

A preprint released this month puts a number on a question most throughput benchmarks never ask: under aggressive KV-cache admission, an SSD holding offloaded KV data can exhaust its rated endurance within days to months. That figure comes from a single, non-replicated preprint, Gandalf, and its authors are candid that their simulator assumes a write-amplification factor of 1 and models no garbage collection or wear leveling, so the true wear rate on real drives is plausibly faster than reported. Treat “days to months” as an optimistic ceiling, not a planning number. The practical consequence stands regardless of the exact rate: if you serve long contexts with KV cache offloaded to NVMe, you are now running a storage-endurance budget, and the drive is a consumable, not a one-time purchase.

Most of the damage appears to be policy-driven. A companion preprint, SARIA, measured that in its reference replay, 82.5% of SSD writes under full duplicate caching never receive a subsequent SSD read. Write volume, and therefore wear, is a knob you control through admission policy, not a fixed tax on offloading.

What the offload path actually writes

The reason flash offload exists at all is arithmetic. KV cache grows with sequence length and, more painfully, with batch. A separate cloud VM-selection study measured an OPT-2.7B model on an AWS g4dn.xlarge with 1,024 input tokens: the KV cache consumed roughly 0.332 GB at batch size 2 and 5.312 GB at batch size 32, a 16x multiplier from batching alone (arXiv:2504.11816).

Once you spill, the spill is large. Gandalf reports that at peak, the concurrently active KV-cache working set in its evaluation reaches 8.6 TB, exceeding what CPU DRAM can economically hold (arXiv:2610.30791). This is the economic logic: NVMe SSDs are roughly 100x cheaper per GB than DRAM, per the Tutti paper, which is why offloading is tempting in the first place. But that 100x figure is a capital-cost comparison. It excludes replacement cost, and replacement cost is exactly what wear introduces.

Why every published wear number is an optimistic bound

Here is the part that should discipline how you quote Gandalf’s headline. Its simulator assumes WAF=1: each byte the host writes is treated as one byte written to flash media. Real SSDs do not behave this way. Garbage collection relocates valid pages before erasing blocks, wear leveling rewrites cold data to even out cell aging, and both add media writes the host never sees. Gandalf states this plainly: “The simulator assumes write-amplification factor (WAF) to be 1, since the simulator does not model the SSD internal operations, such as garbage collection, wear leveling, and other internal operations that may affect the SSD lifetime,” and concludes that the simulator “may overestimate the SSD lifetime for all the evaluated techniques, Gandalf and baselines” (arXiv:2610.30791).

Assuming real wear is faster than the simulated rate is therefore conservative engineering, not speculation. The direction of the error is known. What is not known is the magnitude, because no workload-specific WAF measurement appears in either Gandalf or SARIA. Streaming, append-heavy write patterns like KV persistence can sometimes keep WAF low, but that is a general storage property, not something measured in either paper, so do not bank on it.

The never-read write problem

The most actionable finding in this cluster of papers is SARIA’s: in its reference replay, 82.5% of SSD writes under full duplicate caching receive no subsequent SSD read (arXiv:2610.31169v1). Four out of five bytes wearing the drive bought nothing.

Two cautions apply. First, this is one agent reference-replay workload, not a universal law of KV offloading; your traffic mix will differ. Second, the number measures utility, not necessity, and it is load-dependent: in SARIA’s own replay (arXiv:2610.31169v1) the never-read fraction falls from 82.5% under lower load to 44.3% under higher load, as DRAM contention pushes more reuse past memory residency. The right response is to measure the same ratio on your own workload rather than adopt the headline figure as a design constant. But the mechanism is general: admission policies that persist KV eagerly, including duplicate copies, convert cache-hit optimism directly into drive wear. If you can only fix one thing before enabling offload, cap admission of KV unlikely to be re-read. That is an inference from SARIA’s measurement, not a validated production recipe, and it is the most effective change the evidence supports.

The endurance math you have to do yourself

Gandalf and SARIA anchor their endurance math on the same drive specification: standard-endurance enterprise TLC rated at 1 DWPD over five years. SARIA states it directly, “We consider standard-endurance TLC SSDs rated at 1 DWPD over five years,” citing a KIOXIA CD8P-R product brief (arXiv:2610.31169v1), and Gandalf assumes “1 DWPD (Drive Writes Per Day) for 5 years, which is a majority specification for enterprise SSDs” (arXiv:2610.30791). Neither paper compares drive classes or provides model-by-model datasheet guidance, and that gap is where the buy decision actually lives. The method, however, is straightforward and durable:

Measure host bytes written per day on the offload path under your real admission policy, at your real batch sizes and context lengths.
Divide the drive’s rated endurance (TBW from the vendor datasheet) by that daily figure, then apply a WAF above 1 to get media writes. Since neither paper reports a measured WAF, treat the unadjusted result as the best case and ask your drive vendor or test with smartctl media-wear counters what realistic WAF looks like for sequential-heavy patterns.
Compare against replacement economics: drive cost, plus the labor and disruption of swapping a worn drive in a serving node, against the cache-hit benefit the offload buys.
Verify TLC versus QLC ratings against current datasheets before purchasing. Neither paper compares drive classes, and quoting remembered endurance ratings would be inventing data.

If that math yields a drive lifetime shorter than your hardware refresh cycle, the offload is costing you money the throughput benchmarks never showed.

When flash offload stops being viable

Offload is a means to hold a working set that does not fit elsewhere. The evidence offers several exits:

Smaller working sets. If shorter contexts or smaller batches keep the active KV set within HBM or affordable CPU DRAM, skip flash entirely. The batching multiplier from the VM-selection study shows how quickly concurrency grows the working set; a workload whose active KV set still fits in memory has no wear problem to manage.
Bandwidth ceilings. Offload reads flow through drives that are slower than the memory hierarchy they imitate. The py-kvcache characterization measured 13.5 GB/s on a Kioxia drive and 6 GB/s on a Samsung PM9A3 with fio, against 25–26 GB/s CPU-to-GPU bandwidth over a 16x PCIe 4.0 link. Reads that miss DRAM and hit the drive ride the slower number.
Tail-latency contamination. Block I/O threads share last-level cache with inference threads; isolating that contention alone lowered P99 TTFT by up to 44% for a prefix-reuse workload in the CXL-SSD study. If you enable offload, cache partitioning or core isolation is part of the cost.
Per-request overhead. In LMCache, the newer O_DIRECT path makes storage traffic explicit and stable, but on short repeated prompts it pays that cost on every request, raising warm TTFT from 27 ms to 93 ms (arXiv:2610.21249).
Emerging hardware. High-Bandwidth Flash offers over 10x higher per-stack capacity than HBM4 (512 GB versus 36 GB) while approaching HBM-class read bandwidth at 1.6 TB/s per stack, according to the write-aware KV cache policy paper. If that hardware reaches your procurement channel, the conventional-SSD offload calculus, including its wear problem, changes. That is a forward-looking vendor-adjacent claim, not something you can buy against today.

Monitoring checklist

If you do enable offload, these are the signals the evidence points at:

Host bytes written per day on the offload device, trended against your admission policy changes. This is the input to every endurance projection.
Read-after-write ratio for persisted KV, per SARIA’s method. A falling ratio means you are paying wear for hits that never come.
Media-wear indicators from the drive (SMART wear counters, percentage used), which capture the WAF the simulator ignored.
P99 TTFT, watched for the block-I/O cache contention the CXL-SSD study quantified; a regression after enabling offload may be CPU-side, not drive-side.

Verdict

Enable flash KV-cache offload only after you have sized it as an endurance problem. Budget measured host writes per day against the specific drive’s rated endurance with WAF above 1, cap admission of KV unlikely to be re-read, and verify drive class against vendor datasheets rather than any number in this article. If the active working set fits in HBM or affordable DRAM at your context lengths, skip flash and the whole wear question with it.

Two limitations should travel with any number you quote from this piece. The days-to-months exhaustion rate rests on one non-replicated preprint whose own simulator disclaims SSD-internal wear modeling, and no independent measurement of real production drive wear-out exists in the evidence. The closest cross-check is SARIA’s estimate that a 4-TB drive rated at 1 DWPD consumes its five-year write budget in about 70 days of continuous full duplicate caching, but that is arithmetic from published prefill throughput, not a measured wear-out (arXiv:2610.31169v1). The 82.5% never-read figure is one replayed workload (arXiv:2610.31169v1). Both are strong enough to change how you plan; neither is strong enough to set your replacement calendar.

Frequently Asked Questions

What is the estimated SSD wear rate under aggressive KV-cache offloading?

under aggressive KV-cache admission, an SSD holding offloaded KV data can exhaust its rated endurance within days to months. That figure comes from a single, non-replicated preprint, Gandalf, and its authors are candid that their simulator assumes a write-amplification factor of 1 and models no garbage collection or wear leveling, so the true wear rate on real drives is plausibly faster than reported.

What percentage of SSD writes under full duplicate caching are never read?

in its reference replay, 82.5% of SSD writes under full duplicate caching never receive a subsequent SSD read. Write volume, and therefore wear, is a knob you control through admission policy, not a fixed tax on offloading.