A preprint released this month puts a number on a question most throughput benchmarks never ask: under aggressive KV-cache admission, an SSD holding offloaded KV data can exhaust its rated endurance within days to months. That figure comes from a single, non-replicated preprint, Gandalf, whose motivation analysis sizes the wear by drive capacity: a 30-TB enterprise SSD exhausts its rated TBW in less than a year, and a 2-TB drive reaches it in just days (arXiv:2610.30791). Its authors are candid that their simulator assumes a write-amplification factor of 1 and models no garbage collection or wear leveling, so the true wear rate on real drives is plausibly faster than reported. Treat “days to months” as an optimistic ceiling, not a planning number.
The same preprint also reports a mitigation: an admission policy that prices every SSD write against its expected reuse extended simulated drive life by 7.3x on average, with little hit-rate cost (arXiv:2610.30791). That result shares the same single-preprint basis, so it deserves the same discipline as the wear figure. But it changes the practical question. If you serve long contexts with KV cache offloaded to NVMe, you are running a storage-endurance budget, and you now have two levers on it: what you write and what you buy.
Most of the damage appears to be policy-driven. A companion preprint, SARIA, measured that in its reference replay, 82.5% of SSD writes under full duplicate caching never receive a subsequent SSD read (arXiv:2610.31169v1). Write volume, and therefore wear, is a knob you control through admission policy, not a fixed tax on offloading.
What the offload path actually writes
The reason flash offload exists at all is arithmetic. KV cache grows with sequence length and, more painfully, with batch. A separate cloud VM-selection study analyzes KV-cache size growth across different models in response to increasing batch sizes, and runs OPT-2.7B on an AWS g4dn.xlarge at a 512-token input length to compare cost efficiency per instance (arXiv:2504.11816). Batching multiplies whatever per-request footprint your model produces, and that multiplication is what sends serving stacks looking for a tier below DRAM.
Once you spill, the spill is large. Gandalf’s analysis of two one-day SemiAnalysis Claude Code traces estimates that representative hybrid models generate 81–240 TB of KV-cache state per day, and that at peak the concurrently active working set reaches 8.6 TB, exceeding what CPU DRAM can economically hold (arXiv:2610.30791). This is the economic logic: NVMe SSDs are roughly 100x cheaper per GB than DRAM, per the Tutti paper, which is why offloading is tempting in the first place. But that 100x is a per-GB price ratio computed from hourly cloud rates, $0.0088 per GB-hour for DRAM against $0.000082 for NVMe in Tutti’s setup. It prices storage as if it never wears out, and replacement cost is exactly what wear introduces.
Why every published wear number is an optimistic bound
Here is the part that should discipline how you quote Gandalf’s headline. Its simulator assumes WAF=1: each byte the host writes is treated as one byte written to flash media. Real SSDs do not behave this way. Garbage collection relocates valid pages before erasing blocks, wear leveling rewrites cold data to even out cell aging, and both add media writes the host never sees. Gandalf states this plainly: “The simulator assumes write-amplification factor (WAF) to be 1, since the simulator does not model the SSD internal operations, such as garbage collection, wear leveling, and other internal operations that may affect the SSD lifetime,” and concludes that the simulator “may overestimate the SSD lifetime for all the evaluated techniques, Gandalf and baselines” (arXiv:2610.30791).
Assuming real wear is faster than the simulated rate is therefore conservative engineering, not speculation. The direction of the error is known. What is not known is the magnitude, because no workload-specific WAF measurement appears in either Gandalf or SARIA. Streaming, append-heavy write patterns like KV persistence can sometimes keep WAF low, but that is a general storage property, not something measured in either paper, so do not bank on it.
The same caveat applies to the lifetime-gain numbers below. WAF=1 flatters the absolute lifetimes of every policy Gandalf evaluates, the mitigation and the baselines alike, so the ratio between policies is the more portable output. The authors argue that cutting host-write volume should also reduce WAF itself; that is their inference, not a measurement.
The never-read write problem
The most actionable finding in this cluster of papers is SARIA’s: in its reference replay, 82.5% of SSD writes under full duplicate caching receive no subsequent SSD read (arXiv:2610.31169v1). Four out of five bytes wearing the drive bought nothing.
Two cautions apply. First, this is one agent reference-replay workload, not a universal law of KV offloading; your traffic mix will differ. Second, the number measures utility, not necessity, and it is load-dependent: in SARIA’s own replay the never-read fraction falls from 82.5% under lower load to 44.3% under higher load, as DRAM contention pushes more reuse past memory residency (arXiv:2610.31169v1). But the mechanism is general: admission policies that persist KV eagerly, including duplicate copies, convert cache-hit optimism directly into drive wear.
What Gandalf adds: writes you can decline
Gandalf’s mitigation exploits a property of hybrid model architectures. Modern hybrid LLMs combine full-attention layers with linear- or sliding-window-attention layers, and the two produce state with very different reuse: full-attention KV entries for the whole prefix are re-read at every conversation round, while a linear-attention state is typically used once, to generate the next round, and then superseded (arXiv:2610.30791). Persisting a one-use state to SSD consumes endurance and capacity for a hit that never arrives there. Gandalf therefore keeps linear-attention states in DRAM and full-attention KV on SSD, and gates SSD admission with an explicit cost model: a write is admitted only when its expected recompute savings exceed the write cost plus the value of whatever it evicts.
The reported results are large. Across three traces, two production-derived and one benchmark-generated, with a hybrid model carrying 72% linear-attention layers, Gandalf wrote 2.0–6.5x fewer bytes than the lowest-miss baseline and, at its lowest miss ratio, extended simulated SSD lifetime by 2x on the Aliyun trace, 3.3x on the Claude Code trace, and 6.5x on the AIPerf trace (arXiv:2610.30791). The paper separately reports 7.3x on average, a summary across the models and hardware capacities it evaluates, with maintained or reduced miss ratios; that figure is not the mean of the three trace-level gains. In a Claude Code configuration with Qwen3.5-27B, it wrote 92 TB where an LRU-K baseline wrote 153 TB, stretching simulated life from 59 to 98 days at an essentially unchanged miss ratio of 0.099 versus 0.098 (arXiv:2610.30791).
The cost accounting is the more provocative number for a platform budget. Once SSD wear is priced as a share of drive purchase cost, wear made up 66–86% of per-request cost under the LRU, Marconi, and LRU-K baselines in the authors’ Claude Code setup, and Gandalf came in at $0.30 per 1k requests against $0.74 for the cheapest baseline (arXiv:2610.30791). Those dollars rest on assumed prices inside a simulator, so read them as a claim about where the money goes, not a quote for your fleet. The latency price in the same setup was modest: 0.78s average cache-load latency against 0.66s for the fastest baselines, in exchange for 17.0 versus roughly 4 months of simulated drive life (arXiv:2610.30791).
Honesty requires the other side of the trade. On the Aliyun trace at its default setting, Gandalf missed more than plain LRU (0.818 versus 0.697) in exchange for a simulated 73x longer drive life, because dropping linear states forces full recomputation when a conversation returns after its state left DRAM (arXiv:2610.30791). The policy exposes this as a tunable knob, and the price of turning it varies by workload: raising the write-selectivity parameter over its full range grew lifetime 6.5x on the Aliyun trace but only 2.3x on Claude Code, while the miss ratio rose by 0.016 on the former and 0.071 on the latter (arXiv:2610.30791). Same knob, different bill.
How far the 7.3x result travels
Three boundaries matter before you plan around these numbers.
Model architecture. Every model Gandalf evaluates is hybrid. The paper itself says dense models, full attention across all layers, present a near one-for-one trade-off between SSD writes and cache hits, because every admitted entry carries comparable reuse value (arXiv:2610.30791). The lifetime gain grew with the ratio of linear-state size to full-attention KV size across the four tested hybrids, from 147x to 1,016x, and even the smallest-ratio model still gained 3.9x (arXiv:2610.30791). If you serve dense models, the attention-type signal does not exist, and what remains is the generic capacity-aware admission gate, whose standalone value the paper does not isolate.
Workload class. These traces are multi-round conversation and agentic workloads, where the one-use-then-discard pattern holds. A separate evaluation of KV offloading on context-intensive tasks found significant accuracy drops from methods that looked safe on needle-style retrieval benchmarks (arXiv:2604.08426). That is an accuracy failure rather than a wear failure, but the lesson is the same: offloading behavior validated on one workload class does not automatically transfer to another.
Evidence base. This is one preprint, author-reported, simulated with WAF=1, implemented on LMCache and vLLM, with no independent replication. The gains may not transfer across drive models, filesystems, or offloaders with different write paths. Strong enough to justify an experiment on your own traces; not strong enough to cancel a procurement decision.
The endurance math you have to do yourself
Gandalf and SARIA anchor their endurance math on the same rating: 1 DWPD sustained over five years. SARIA is the one that names the drive class, stating “We consider standard-endurance TLC SSDs rated at 1 DWPD over five years,” citing a KIOXIA CD8P-R product brief (arXiv:2610.31169v1); Gandalf assumes “1 DWPD (Drive Writes Per Day) for 5 years, which is a majority specification for enterprise SSDs” without specifying a media type (arXiv:2610.30791). Neither paper compares drive classes or provides model-by-model datasheet guidance, and that gap is where the buy decision actually lives. The method, however, is straightforward and durable:
- Measure host bytes written per day on the offload path under your real admission policy, at your real batch sizes and context lengths.
- Divide the drive’s rated endurance (TBW from the vendor datasheet) by that daily figure, then apply a WAF above 1 to get media writes. Since neither paper reports a measured WAF, treat the unadjusted result as the best case and ask your drive vendor or test with
smartctlmedia-wear counters what realistic WAF looks like for sequential-heavy patterns. - Re-run the first two steps with admission filtering enabled if your offloader supports it. The cheapest terabyte of endurance is the write you never issue.
- Compare against replacement economics: drive cost, plus the labor and disruption of swapping a worn drive in a serving node, against the cache-hit benefit the offload buys.
- Verify media type and endurance class against current datasheets before purchasing. Neither paper compares drive classes, and quoting remembered endurance ratings would be inventing data.
If that math yields a drive lifetime shorter than your hardware refresh cycle, the offload is costing you money the throughput benchmarks never showed.
Filtering, bigger drives, or more memory
The choice between fixing the writer and buying endurance depends on your write mix, your drive budget, and what a replacement visit costs you.
On write mix: measure your never-read fraction before buying anything. If it resembles the 82.5% lower-load figure from SARIA’s replay (arXiv:2610.31169v1), admission filtering is nearly free endurance. SARIA’s eviction-count gate cut SSD writes by 57.1–58.9% relative to full duplicate caching (arXiv:2610.31169v1), and frequency-threshold admission already ships in production offloaders: NVIDIA Dynamo persists a block only after its access count reaches two, per Gandalf’s citation of the KVBM guide (arXiv:2610.30791). If your models are hybrid and your offloader can key admission on attention type, the reported gains are large enough that software should be evaluated before hardware.
On endurance rating: neither paper models anything beyond 1 DWPD, so they cannot tell you what a higher-endurance class is worth. What they do show is that solving wear with raw capacity pays poorly. In SARIA’s replay, growing the cache from 3M to 32M token slots, the capacity needed for five-year endurance under its write rate, raised the cache hit rate only from 82.0% to 84.8% (arXiv:2610.31169v1). Capacity bought for endurance buys little cache. Gandalf’s related-work survey likewise notes that spreading writes across more drives raises aggregate endurance only at added acquisition, power, and maintenance cost (arXiv:2610.30791). A higher-DWPD drive class is a different purchase, a straight multiplier on the write budget at a price premium neither paper quantifies.
On replacement labor: a 4-TB drive at 1 DWPD that continuous full duplicate caching exhausts in about 70 days, SARIA’s arithmetic from published prefill throughput (arXiv:2610.31169v1), is a quarterly maintenance event, with the drain, swap, and re-warm work that implies. If filtering stretches that past a year, replacement folds into the normal refresh cycle, and the comparison shifts back to what offload was always about: NVMe capacity versus GPU memory. That reframing is inference from the papers’ numbers, not a result they report, but it is the right frame for the SKU decision. If offloader-side filtering holds up on your traces, the bottleneck moves back to HBM and DRAM pressure, and larger-memory nodes compete against cheap flash again on throughput rather than on drive mortality.
When flash offload stops being viable
Offload is a means to hold a working set that does not fit elsewhere. The evidence offers several exits:
- Smaller working sets. If shorter contexts or smaller batches keep the active KV set within HBM or affordable CPU DRAM, skip flash entirely. The VM-selection study measures how KV-cache size grows with batch size; a workload whose active KV set still fits in memory has no wear problem to manage.
- Bandwidth ceilings. Offload reads flow through drives that are slower than the memory hierarchy they imitate. The py-kvcache characterization measured 13.5 GB/s on a Kioxia drive and 6 GB/s on a Samsung PM9A3 with fio, against 25–26 GB/s CPU-to-GPU bandwidth over a 16x PCIe 4.0 link. Reads that miss DRAM and hit the drive ride the slower number.
- Tail-latency contamination. Block I/O threads share last-level cache with inference threads; isolating that contention alone lowered P99 TTFT by up to 44% for a prefix-reuse workload in the CXL-SSD study. If you enable offload, cache partitioning or core isolation is part of the cost.
- Per-request overhead. In LMCache, the newer O_DIRECT path makes storage traffic explicit and stable, but on short repeated prompts it pays that cost on every request, raising warm TTFT from 27 ms to 93 ms (arXiv:2610.21249).
- Emerging hardware. High-Bandwidth Flash offers over 10x higher per-stack capacity than HBM4 (512 GB versus 36 GB) while approaching HBM-class read bandwidth at 1.6 TB/s per stack, according to the write-aware KV cache policy paper. Gandalf’s authors note their placement policy should extend to flash tiers like HBF (arXiv:2610.30791). If that hardware reaches your procurement channel, the conventional-SSD offload calculus changes. That is a forward-looking claim, not something you can buy against today.
Monitoring checklist
If you do enable offload, these are the signals the evidence points at:
- Host bytes written per day on the offload device, trended against your admission policy changes. This is the input to every endurance projection.
- Read-after-write ratio for persisted KV, per SARIA’s method (arXiv:2610.31169v1). A falling ratio means you are paying wear for hits that never come.
- Miss-ratio cost of any admission threshold you set. Gandalf’s sweep shows the same filter cost between 0.016 and 0.172 additional misses depending on the trace (arXiv:2610.30791), so tune on your traffic, not on theirs.
- Media-wear indicators from the drive (SMART wear counters, percentage used), which capture the WAF the simulator ignored.
- P99 TTFT, watched for the block-I/O cache contention the CXL-SSD study quantified; a regression after enabling offload may be CPU-side, not drive-side.
Verdict
Enable flash KV-cache offload only after you have sized it as an endurance problem, and now, after you have checked whether your serving stack can decline the writes that never get read. Budget measured host writes per day against the specific drive’s rated endurance with WAF above 1, and cap admission of KV unlikely to be re-read. If your models are hybrid and your offloader supports attention-aware or eviction-history admission, the reported lifetime gains are large enough to evaluate before you spend on higher-endurance drives. If your models are dense, or your workload defeats reuse prediction, the hardware lever is the one the current evidence supports. If the active working set fits in HBM or affordable DRAM at your context lengths, skip flash and the whole wear question with it.
Two limitations should travel with any number you quote from this piece. The days-to-months exhaustion rate and the 7.3x lifetime gain both rest on one non-replicated preprint whose own simulator disclaims SSD-internal wear modeling, and no independent measurement of real production drive wear-out exists in the evidence (arXiv:2610.30791). The closest cross-check is SARIA’s estimate that a 4-TB drive rated at 1 DWPD consumes its five-year write budget in about 70 days of continuous full duplicate caching, but that is arithmetic from published prefill throughput, not a measured wear-out (arXiv:2610.31169v1). The 82.5% never-read figure is one replayed workload (arXiv:2610.31169v1). These results are strong enough to change how you plan and what you test; none is strong enough to set your replacement calendar.

Join the discussion
Share a useful perspective or ask a question about this article.