A cloud-local SSD begins with an attractive physical fact: the drive sits in the compute server, so an I/O request does not cross the data-center network. That fact does not guarantee local-device performance. Alibaba’s production history shows that the limiting resource moved repeatedly even though the SSD remained in the same chassis[1]. A kernel path first spent too many cycles on context changes and interrupts. A user-space polling path recovered throughput but reserved CPU cores. An ASIC DPU removed those cores but could not keep pace with later drives or new storage functions. A combined ASIC and Arm SoC restored programmability while keeping the common path in hardware.
The FAST 2026 paper is valuable because it records this sequence across systems deployed at thousands to tens of thousands of servers. It also separates the remaining problem from the solved one. RISTRETTO, the third generation, makes a virtual disk perform close to a physical PCIe Gen4 SSD. It does not give a local disk remote durability, fine-grained elasticity, or independence from a compute-node failure. LATTE, the paper’s proof of concept, therefore combines the local device with standard elastic block storage. The result is not a claim that every cloud disk should become a cache. It is evidence that the datapath and the failure contract must be designed together.
The first bottleneck was software, not NAND
Alibaba’s early local-storage stack had been designed around hard drives. A virtual machine submitted requests through virtio and KVM, the host kernel issued device I/O, and completions returned through another sequence of interrupts and transitions. That path was acceptable when the medium was slow. With a PCIe Gen3 NVMe SSD, the paper measures only 9.54% of the device’s maximum IOPS while consuming as much as 1.4 CPU cores. Short media latency exposed every trip through the virtualization and kernel path.
ESPRESSO moved request processing to an SPDK-based user-space service and assigned polling threads to virtual disks. A 12-drive server reached 38.4 GB/s and 5.76 million IOPS, or 3.2 GB/s and 480,000 IOPS per drive. The production system launched around 2017 and expanded to tens of thousands of servers. Against the earlier HDD-oriented stack, reported software overhead fell by 82.35%.
Polling did not make the work free. Each thread was bound to a core, yet field measurements placed the 99th-percentile active utilization below 60%. Those apparently idle cycles could not be safely reclaimed because an I/O burst might arrive without warning. ESPRESSO also needed host CPUs and therefore conflicted with bare-metal instances. A completion still crossed the VM boundary, adding 5 to 12 microseconds in the evaluated path. The system had converted latency overhead into reserved compute capacity.
This is the first procurement lesson. A local-storage price should include the cores and memory that the virtualization path removes from saleable compute. A drive with higher IOPS can make a server less efficient when software requires more polling cores to expose that performance. Device specifications and instance economics can move in opposite directions.
DPU offload removed cores and introduced a new ceiling
DOPPIO placed the virtualized I/O path on commercial ASIC DPUs. Each DPU controlled two attached SSDs, accepted requests from the host, applied fixed hardware functions, and generated completion interrupts without running the datapath on host CPUs. Six DPUs and twelve PCIe Gen3 drives delivered as much as 38.4 GB/s and 6 million IOPS. The design also allowed bare-metal compute because storage no longer reserved the host cores used by ESPRESSO.
The bottleneck then moved into the offload device. The paper reports that the selected ASIC DPU could sustain roughly 500,000 IOPS per attached SSD while 2023 drives had advanced toward 1.5 million read IOPS. Fixed hardware also made an emerging function such as logical-volume management difficult to add. A larger FPGA could supply flexibility, but the authors rejected that path on power and capital-cost grounds.
This transition exposes a general DPU tradeoff. Offload is beneficial when a stable, repetitive datapath consumes expensive general-purpose cores. The same specialization can strand new media capability or delay a control-plane feature. An operator should therefore ask how much of the path is fixed, which functions remain programmable, and whether the offload device has headroom for the next SSD generation. The answer matters more than whether the component is marketed as a DPU, SmartNIC, or storage accelerator.
RISTRETTO split the fast path from the changing path
RISTRETTO is a PCIe extension card that combines an ASIC with an Arm SoC containing four Cortex-A72 cores and 64 GB of DRAM. The ASIC presents virtual NVMe controllers, moves commands and data, and handles the predictable fast path. Software on the SoC implements the block abstraction and functions that are likely to change. This division keeps common operations in efficient logic while avoiding the all-or-nothing programmability decision that limited DOPPIO.
The largest reported RISTRETTO instance uses eight virtual disks and reaches 7.2 million IOPS. A single virtual disk delivers 900,000 read IOPS and 6.7 GB/s, compared with 1 million IOPS and 6.9 GB/s for the underlying physical-drive example. Stable-state write performance is identical at 180,000 IOPS in the paper’s table, while write throughput differs by 0.1 GB/s. The small gap is the point: the virtualization layer is no longer the dominant loss.
Deployment began in 2023 and had reached several thousand nodes by the paper’s cutoff. The platform can expose an entire SSD or a partition through SR-IOV, and the SoC software can change queue mappings and storage functions without redesigning the ASIC. In architectural terms, RISTRETTO assigns mechanism by change rate. Packet movement and virtual-controller behavior belong in fixed logic; policy and evolving block services remain in software.
The measured result should not be read as proof that an ASIC-SoC card is universally necessary. A cloud with lower IOPS targets may accept host software. A server with spare cores may prefer the operational simplicity of SPDK. The design becomes compelling when host cores are revenue-bearing, bare-metal service is required, and the fleet is large enough to amortize custom hardware and firmware.
Near-physical performance does not repair a failed host
Once RISTRETTO removes most datapath overhead, local storage’s original disadvantages become visible again. Capacity is allocated at device or partition granularity. The data is attached to one compute node. Applications must create their own replicas or accept loss after a disk or node failure. The paper cites a local-disk annual failure rate around 0.44%, but the larger operational concern is recovery coupling: compute and data can disappear together, and replacement capacity may not be available on the same schedule as the application.
LLM systems sharpen this conflict. Local flash can hold model parameters, checkpoints, or KV-cache state at a favorable price and latency. Dynamic scaling needs the same data on another node. A replicated application can solve the problem, but three copies can erase the media-cost advantage and force every software team to implement placement and repair.
Standard elastic block storage provides a different contract. Alibaba’s high-end EBS can reach 1 million IOPS and about 30 microseconds according to the paper, but it costs more because remote access, redundancy, and elasticity are included. The right comparison is therefore not local latency against remote latency. It is the complete cost of providing the same availability, recoverability, and scale-out behavior.
LATTE makes the remote copy authoritative
LATTE combines one RISTRETTO-backed local cache with a more affordable standard EBS volume. Writes can enter the cache or bypass it, then eventually reach the backend. Reads check local mapping state and fetch from EBS when a block is absent. A learned dispatcher observes recent I/O behavior and congestion to choose a path, while admission and eviction logic avoids filling the cache with blocks accessed only once.
The design reports an admission hit rate above 82% for its evaluated traces. At a 75% read-hit setting, the combined system exceeds the IOPS of RISTRETTO alone because cache and backend bandwidth can serve requests concurrently. At 100% hits, the backend is idle and the result returns to the local-device ceiling. This non-monotonic observation is important: the hybrid wins by scheduling two resources, not because a cache hit is somehow faster than the local SSD.
The cost model normalizes 4 TB of RISTRETTO capacity to one unit. A maximum-provisioned LATTE configuration is assigned 13 units, while an automatically provisioned version ranges from 2.1 to 4.0. A high-performance EBS comparison reaches about 19 because its software stack and three-way replication are included. These values are paper-specific estimates, not public cloud prices. They show the variables that move the result: remote bandwidth, replication, cache hit rate, and the amount of local capacity reserved.
Most importantly, LATTE is explicitly a proof of concept. The authors identify simultaneous failure handling and more efficient backend redundancy as future work. Publication does not establish fleet-scale durability, multi-tenant isolation, or long-duration behavior of the learned dispatcher. The hybrid architecture is a direction supported by a prototype, whereas ESPRESSO, DOPPIO, and RISTRETTO include production histories.
A learned path selector needs a deterministic fallback
Machine learning is not the architectural center of LATTE. The durable idea is that local and remote paths can be used concurrently under one block interface. The learned component estimates which path is preferable from a short window of recent operations and observed latency. Any deployment still needs a deterministic safe path when the model is stale, unavailable, or exposed to a workload transition.
The fallback should preserve three invariants. An acknowledged durable write must remain recoverable from the remote tier. Ordering must survive movement between cache and backend. A cache miss or wrong prediction may cost latency but must not change data. Telemetry should record why each request used local cache or EBS, the queue state at that moment, and when the remote copy became authoritative.
This instrumentation also prevents a good average from hiding a bad tail. A dispatcher can improve mean latency by routing most requests well while sending a small latency-sensitive class into congestion. Evaluation should separate hit latency, miss latency, write acknowledgment, remote persistence, and recovery after local loss. Those denominators decide whether the hybrid is suitable for model loading, temporary analytics data, databases, or checkpoints.
The next SSD generation should trigger a control-path audit
Alibaba’s four designs form a practical audit sequence. When media performance rises, first determine whether kernel transitions dominate. If user-space polling fixes the problem, price the dedicated cores. If those cores are unacceptable, consider offload, then measure whether its fixed logic and processing rate leave headroom. If near-physical access is achieved, stop optimizing the datapath and examine the availability contract.
The sequence avoids a common mistake: applying the last generation’s solution to the next generation’s bottleneck. ESPRESSO solved context-switch overhead, DOPPIO solved host-core use, RISTRETTO solved DPU programmability and throughput, and LATTE addresses the durability and elasticity that no local datapath can provide alone. Each step changes the scarce resource.
For an infrastructure buyer, the paper suggests six numbers to place beside SSD IOPS: host cores reserved per drive, virtual-to-physical latency gap, offload-device ceiling, smallest allocatable capacity, time to restore service after node loss, and cost of the authoritative remote copy. If these values are absent, “near physical” describes only the fastest part of an incomplete system.
Source and copyright notice
This article is an independent editorial digest prepared by Silicon & Systems from the authors’ public FAST paper. It restates the mechanisms, measurements, deployment history, and limitations in original language. No source sentence, table, or publisher figure is reproduced. The hardware plate and thumbnail were created for this article from reported facts and a generic server material plate. Copyright for the paper remains with its authors. The full paper is available from the official USENIX FAST 2026 page.