Identical text does not guarantee a reusable KV cache. In an agent workflow, a system instruction, accumulated history, or memory segment can remain byte-for-byte unchanged while a new tool result or reasoning step appears before it. The stable segment then moves to different absolute token positions. Since positional encoding contributes to every key and value, loading yesterday’s representation at today’s location can change attention even though the text is the same.
The FAST 2026 CacheSlide paper identifies this as a third reuse regime between fixed-prefix caching and position-independent caching[1]. Its term is Relative-Position-Dependent Caching (RPDC): reusable segments keep their order, while the mutable spans between them change length. CacheSlide attempts to preserve the attention already captured among stable segments, repair only the interaction with changed spans, and prevent the repair path from turning SSD spill into a new bottleneck.
The headline results are substantial. Across the evaluated models and agent benchmarks, the paper reports 3.11 to 4.3 times lower latency and 3.5 to 5.8 times higher throughput than selected baselines. A more informative comparison holds output quality closer: against ContextCache, CacheSlide lowers time to first token by 2.4 to 3.3 times with negligible measured accuracy loss. Those numbers do not describe an unmodified drop-in cache. CacheSlide changes positional encoding through adapter-based continued training, uses a workload template, and tunes how many tokens are corrected. The system contribution and those operating conditions must be evaluated together.
Agent prompts create a third caching regime
Prefix caching works when every reusable token begins at the same position and all new material follows it. Agent prompts often violate that geometry. A memory-oriented agent can keep a system prefix and a long history while inserting a changing working window between them. A coding agent may preserve instructions and prior tool output while replacing several control fields. Moving those fields to manufacture a common prefix can alter recency emphasis or break dependencies among prompt components.
Position-independent schemes accept arbitrary placement, but they reset or otherwise abstract the cached segment’s coordinates. They then recompute a fraction of tokens to recover attention that the changed position disturbed. The problem is that attention heads do not all depend on the same tokens, and the most consequential subset is not always knowable before decoding. More correction protects quality but removes the compute advantage; less correction can leave a silent output-quality error.
RPDC narrows the problem. It assumes stable chunks remain in the same relative sequence, even though their absolute coordinates move because updated chunks between them grow or shrink. That assumption matches several evaluated agent templates and allows CacheSlide to retain interactions within one stable chunk and across stable chunks. Only attention involving the changed material should need repair. The distinction matters operationally: a platform should classify prompt structure before selecting a cache, rather than treating every repeated substring as equivalent reusable state.
The paper quantifies why position matters. For shifts from 0 to 1,000 tokens in one MemGPT experiment, similarity between cached and recomputed keys falls by more than 90% with RoPE, while the evaluated CoPE configuration falls by 28%. A padding workaround also fails to provide a general answer. Fixing mutable windows at 1K, 2K, or 3K tokens reduces F1 by more than 78.1% relative to the no-padding baseline at the same reasoning rounds in the reported test. A small window can discard necessary context; a large one still moves later segments and spends compute on padding.
CCPE makes movement smaller in the model’s coordinate system
Chunked Contextual Position Encoding (CCPE) divides one task template into reusable and recomputed chunks. During task-specific preparation, it observes CoPE encodings for prompts of that class and records the most frequent positional pattern for the reusable regions. A later request uses those learned ranges for stable chunks and computes current positions for mutable chunks. Content hashes locate the cached KV state.
The important claim is not that position disappears. CCPE tries to reduce the coordinate discrepancy while preserving the coarse order that the agent template requires. CoPE can assign the same contextual position to adjacent tokens around semantic boundaries, so an absolute token shift need not rotate every token representation as RoPE would. If a cached history chunk and its fresh counterpart remain close in this coordinate system, attention within that chunk and among other stable chunks can be reused with less error.
This mechanism adds a deployment dependency. The authors enable CoPE with LoRA adapters learned through continued pretraining while leaving the backbone weights intact. CacheSlide runs with those adapters active, whereas the evaluated baselines retain their native RoPE or ALiBi behavior. Thus, the measured system is not merely a new vLLM eviction policy. A production team needs model-specific adapter validation, a prompt-template version, and a fallback for tasks that do not exhibit the learned pattern.
The template boundary is also narrower than general semantic caching. CCPE does not prove that two paraphrases share a valid KV representation. It relies on identical reusable content and a recurring arrangement of chunks. Dynamic agent planners that reorder tools, synthesize new schemas, or branch into task-specific prompt layouts can fall outside the RPDC assumption even when much of their text repeats.
Weighted correction spends compute where the cached state moved most
Position alignment alone does not recover attention between stable and updated chunks. Weighted Correction Attention (WCA) addresses that remaining boundary. In the first layer it recomputes the prompt, measures the difference between cached and fresh keys, and selects the tokens with the largest deviations. In later layers it recomputes only the selected subset and blends fresh and cached KVs with deviation-dependent weights.
Every four layers, WCA checks similarity again. A token whose corrected representation has converged can leave the active set, allowing another high-discrepancy token to enter. The design uses increasing similarity across adjacent deeper layers as evidence that repair can stop. In the reported parameter sweep, quality-adjusted throughput peaks near a selected fraction of 0.26 and a CKSim threshold near 0.12 across two tested settings.
Those values are empirical controls, not constants of transformer architecture. The optimum depends on model depth, attention behavior, prompt composition, and the quality metric. An operator should expose the selected fraction, convergence rate, and per-task quality regression rather than hiding them inside a global cache switch. A throughput increase is only useful when the denominator counts correctly completed work.
The method also pays a full first-layer recomputation to learn where repair is needed. For short prompts or templates with little reusable material, that fixed cost can erase the benefit. CacheSlide is most plausible when stable chunks dominate, requests repeat enough to amortize cache creation and adapter preparation, and first-token latency matters more than maintaining the simplest model-serving path.
SLIDE treats KV correction as a storage scheduling problem
Selected tokens introduce layer-wise reads and writes. A conventional serving runtime loads the old KV page and then writes the updated values, serializing two operations on the prefill path. If capacity pressure spills pages to NVMe, partially modified pages create small random writes and amplify device traffic. A cache algorithm that saves attention compute can therefore lose its gain in the memory hierarchy.
SLIDE extends vLLM 0.8.5 with extra pages for corrected tokens. When recomputation finishes before the old page loads, the new values can be written elsewhere instead of waiting. During decoding, the system overwrites the original slots when possible to recover space. It marks pages containing selected tokens as dirty, evicts clean pages first, and groups unavoidable dirty writeback by the number of corrected tokens to create larger sequential operations.
The ablation makes this storage path visible. Layer-wise load-write decoupling reduces the measured parallel latency by 26.7% to 51.5% as batch size rises from 2 to 6. The dirty-page mechanism reduces write stalls by 66.9% to 73.5% across the reported batch-size sweep. SSD write amplification falls by 3.11 to 3.62 times, and GPU memory use is 1.63 to 1.9 times lower than PromptCache in the corresponding comparison. These are separate ablations and should not be multiplied into a synthetic end-to-end number.
Clean-first eviction is not automatically fair under multiple tenants. A request that has many corrected pages can retain them while another request loses clean reusable pages. The paper focuses on aggregate latency and throughput, not isolation, SSD endurance over a service lifetime, or tail behavior under mixed priorities. A production scheduler would need request-aware limits in addition to page state.
The evaluation measures three agents, three models, and one storage hierarchy
The principal host has 500 GB of DRAM, 2 TB of NVMe storage, and PCIe Gen4 connections. Most experiments use one NVIDIA A100 with 80 GB HBM; Llama-3 70B uses two A100 GPUs. The software stack includes Ubuntu 20.04, Linux 5.16.7, CUDA 12.6, and vLLM 0.8.5. Models include Mistral-7B, MPT-30B, and Llama-3 70B.
The agent coverage is deliberately varied. Reflexion uses HotPotQA for iterative reasoning, MemGPT uses a multi-session conversation corpus, and SWE-Agent uses a 12-repository Python task set. Metrics differ by task, including ROUGE-L recall, success rate, and F1. Time to first token is tested at batch size one for the quality-latency frontier, while parallel and beam-search experiments raise storage pressure.
Against CacheBlend, CacheSlide reports 1.21 to 2.11 times lower TTFT and 1.97 to 2.28 times better task accuracy across the plotted combinations. Relative to PromptCache, the ranges are 1.12 to 2.45 times for TTFT and 1.41 to 3.95 times for accuracy. At batch size eight in the throughput study, the paper reports an average 63.1% throughput increase and 68.9% lower throughput standard deviation than the chosen baselines. These comparisons show an accuracy-latency frontier in the tested agent templates, not a universal ranking over all context-caching systems.
Several scope limits remain. The evaluation uses one server and up to two GPUs, so it does not measure distributed KV movement, disaggregated prefill, or network-attached cache tiers. CoPE adapter training cost and its effect on unrelated tasks need separate accounting. The three agent families support the RPDC premise, but production prompts can change with model upgrades, safety policies, tool schemas, and application experiments. Finally, correctly completed tasks per second is the right direction for evaluation, yet any deployment still needs its own quality set because ROUGE, exact success, and F1 capture different failure modes.
Treat the prompt template as cache metadata
CacheSlide changes the unit of cache validity. A content hash is necessary but no longer sufficient. The reusable state also depends on the model build, positional adapter, ordered chunk template, correction policy, and storage-manager version. Those fields should form a cache namespace so that a prompt or model rollout cannot silently consume incompatible KVs.
Operators should watch more than hit rate. Useful measurements include the fraction of prompt tokens classified as reusable, positional-similarity distribution, corrected-token fraction by layer, quality regressions by task, clean and dirty spill volumes, SSD write amplification, and TTFT tails under concurrency. Template drift can first appear as a rising correction fraction before task accuracy falls, giving the platform an opportunity to invalidate the cache or revert to fresh computation.
The broader systems lesson is that context caching spans model semantics and storage scheduling. Reusing bytes is not enough when their coordinates define computation, and repairing coordinates is not enough when the repaired pages stall on I/O. CacheSlide connects those layers in one design. Its production value depends on whether an agent platform can prove that relative order is stable, maintain the adapted model and template as versioned artifacts, and measure completed-task quality whenever it spends less compute.
Source and copyright notice
This article is an independent editorial analysis of the FAST 2026 paper by Yang Liu, Yunfei Gu, Liqiang Zhang, Chentao Wu, Guangtao Xue, Jie Li, Minyi Guo, Junhao Hu, and Jie Meng. The authors are affiliated with Shanghai Jiao Tong University, Jinan Inspur Data Technology, Peking University, and Huawei Cloud. The source is openly available from the USENIX presentation page. We restated the mechanisms and measurements in our own words and created the figure specifically for this article. No source text, table, or figure is reproduced. Copyright (c) 2026 the paper authors.