An object store holding several exabytes does not improve by changing one storage node. Its economics are determined by how many complete copies cross regional boundaries, how each region repairs a failed drive, and how much latency a client absorbs when data is reconstructed. Apple’s ACOS paper at FAST 2026 documents a decade-long production transition across all three layers[1].
The first generation stored complete copies in two active regions and protected each copy with a local code. ACOS 2.0 instead divides an object into four data segments, derives one XOR segment, and places the five pieces in five regions. Any four reconstruct the object. Each regional segment still receives local protection through a (20,2,2) locally repairable code (LRC). The resulting capacity multiplier falls from 2.40× to 1.50× under the paper’s configurations.
That 37.5% reduction is not free capacity. A normal read may need remote segments, a failed local fragment can trigger more than one repair tier, and a write is durable only after a multi-region sequence completes. ACOS is therefore useful as a record of exchanged obligations. It shows what a storage operator must build after deciding that full regional replicas cost too much.
Two replicas made failure domains easy to reason about
ACOS serves workloads with materially different request mixes. iCloud produces many object reads and deletes. Media services store large audio and video objects whose delivery path can include edge caches. Maps adds another mix of sizes and access rates. At the system boundary, ACOS must therefore support small metadata-sensitive operations, large sequential transfers, and tenant-specific retention behavior without dedicating a separate store to every service.
The first design used two active regions. Each region held a complete encoded copy, so a client could read the whole object from either location. Within a region, ACOS initially used a (12,2,2) LRC and later a (20,2,2) configuration. The notation describes data and parity groupings that allow common failures to be repaired from fewer fragments than a global erasure-code reconstruction.
With the later local code, two regional copies produce a 2.40× replication factor. That overhead bought a clean separation: local disk and server failures stayed inside one region, while losing a region still left a complete object elsewhere. It also let services favor the closest region without first assembling pieces from multiple geographies.
At several exabytes, however, the extra 0.90 units above ACOS 2.0 represent a large fleet. The tradeoff is not only media cost. Every added byte requires racks, power, network movement, repair bandwidth, and operational capacity. ACOS 2.0 changes the outer protection scheme while retaining local LRC because cross-region redundancy and drive-level repair solve different failures.
A five-region code removes complete regional copies
The outer code is deliberately simple. Four segments contain object data and the fifth contains their XOR. If one region is unavailable, the missing segment can be computed from the remaining four. Compared with a more complex Reed-Solomon code, XOR reduces compute cost and keeps the recovery rule transparent. The design assumes that ACOS can operate in five regions with sufficiently independent failure domains and network paths.
Local LRC then protects each of those five segments. Most disk, server, or rack failures should not require traffic from four remote regions. A local repair reads fragments in the same deployment stamp and reconstructs the unavailable piece. Only when local recovery cannot finish does the global layer obtain the other regional segments.
The paper models the five-region design with a mean time to data loss of 1.31 × 10^21 years and 0.0316 seconds of unavailability per year. These are analytical outputs, not measured lifetime guarantees. The calculation assumes independent failures and 99.999% regional availability. Correlated software defects, operator mistakes, dependency outages, or a common network control plane can invalidate independence. The useful conclusion is comparative: under one stated model, XOR across five regions keeps the durability target while lowering the replication factor.
The code also changes the meaning of a read. A complete copy no longer exists at one regional endpoint. ACOS must obtain enough segments, combine them, and return the object while hiding the topology from clients. The paper reports that ACOS 2.0 GET adds about 50 milliseconds relative to ACOS 1.0 because of inter-region access. Capacity savings are therefore partly paid by network distance.
Layered repair keeps the common failure local
ACOS distinguishes a normal read, a degraded local read, and a global reconstruction. Local LRC reconstruction costs about 2 milliseconds at the 90th percentile in the reported environment. A degraded read adds roughly 30 milliseconds through about the median and can reach hundreds of milliseconds higher in the distribution. The system waits up to 500 milliseconds for local parity before falling back to global recovery.
That timeout expresses an operational judgment. Falling back too early multiplies cross-region traffic during a local disturbance. Waiting too long exposes the client to a stalled request. A fixed value also encounters different network and device conditions across time, so operators need per-path observability rather than a single aggregate latency graph.
Global XOR work itself is small in the reported measurements. Computing the missing segment adds about 0.3 milliseconds at the 90th percentile. A normal global reconstruction adds approximately 10 milliseconds to GET latency, while a regional failover can add up to 50 milliseconds above the 60th percentile. This separation matters because the wide-area transfer and failure detection cost more than the XOR arithmetic.
Repair traffic must also be controlled under simultaneous failures. Local recovery reads more fragments than an ordinary GET, and global reconstruction fans out to four regions. The system needs concurrency limits, backpressure, and prioritization so that recovery does not consume the bandwidth required by healthy foreground requests. The paper’s contribution is the complete layered path, not an assertion that erasure coding alone guarantees availability.
Writes require durability without turning every outlier into latency
ACOS places 99.99% of replication inside the synchronous PUT flow. Only 0.01% completes through an asynchronous path. This split keeps the ordinary durability contract strong while providing a way to finish rare writes that encounter a slow or unavailable dependency. The exact boundary is consequential because an acknowledged but incompletely replicated object carries a different loss exposure.
The client and server paths include several latency optimizations. The paper reports up to 60% server-side and 87% client-side reductions, depending on the optimization and request. Bypassing an intermediate load balancer reduces GET latency by 22% at the median, 32% at the 90th percentile, and 26% at the 95th percentile. Those gains show that the topology around the code can matter more than encoding compute.
Metadata prefetch is another example. The system predicts and retrieves information before the primary path proves it is needed. The reported mismatch rate is about 0.001%, which makes speculation attractive at ACOS scale, but every incorrect prefetch still consumes resources. A deployment evaluating the technique should track wasted work as an absolute request rate, not only a small percentage.
These optimizations do not erase the architectural tax. They reduce extra round trips and queueing introduced by a distributed object path. A service comparing designs should therefore measure the optimized five-region system against an optimized two-copy system, and include the operational cost of maintaining direct routes and prefetch correctness.
Moving several exabytes is a system feature, not a maintenance detail
Apple migrated several exabytes from ACOS 1.0 to 2.0 over multiple years without downtime. That interval changes how the design should be judged. The old and new placement schemes had to coexist, clients had to locate either representation, and background movement could not overwhelm production traffic. Migration correctness was as important as steady-state efficiency.
A safe transition needs an authoritative state machine for each object. Copying data, verifying the new segments, switching the metadata pointer, and retiring the old copy must tolerate retries at every boundary. A crash cannot leave two ambiguous owners or remove the last valid representation. Large-scale deletion is especially sensitive because delayed or replayed messages can otherwise revive data that should be gone.
The fleet also needs rate control tied to foreground demand. At exabyte scale, even a small percentage moved per day is substantial network and device load. Progress should be expressed in verified durable bytes, not merely scheduled objects, since large media files and tiny metadata objects consume different resources. ACOS demonstrates that an encoding redesign is incomplete without a migration design of comparable maturity.
Workload mix determines whether the capacity exchange is worthwhile
The paper reports request composition as ratios across representative workloads. iCloud accounts for 52% of GET, 27% of PUT, and 81% of delete activity in the presented mix. Media contributes 11%, 6%, and 1%, while Maps contributes 20%, 5%, and 1%. Miscellaneous services supply the remainder, including 62% of PUT activity. These are request shares rather than byte shares, so they do not directly state which tenant consumes the most capacity or bandwidth.
Read-heavy objects experience the added inter-region GET path repeatedly. Write-heavy objects pay the multi-region placement cost at ingestion. Delete-heavy services test metadata consistency and reclamation. One global encoding policy can still work, but capacity planning needs request counts, byte distributions, tail latency, and retention duration per tenant.
The most favorable case for ACOS 2.0 is a large, durable object estate where a 0.90× reduction in physical capacity dominates tens of milliseconds of additional read latency. A latency-critical, small-object workload close to one region may prefer a local complete copy or a cache above the coded store. The architecture can support both by treating ACOS as the durable layer and placing latency-specific copies elsewhere, but those caches restore some capacity and invalidation cost.
The decision should be audited as three separate budgets
An operator considering a similar transition should maintain a capacity budget, a latency budget, and a failure budget. The capacity budget includes encoded bytes, temporary migration copies, repair headroom, and caches. The latency budget separates normal network distance, local degraded reads, global reconstruction, and failover detection. The failure budget states which combinations are assumed independent and which shared systems can defeat that assumption.
The 1.50× multiplier is compelling only when all three budgets close. If migration requires long-lived duplicate data, short-term capacity can exceed the old design. If regional links are unstable, global recovery can harm tail latency. If the five regions share a control plane, the analytical durability estimate overstates practical independence.
ACOS shows why production storage papers are valuable beyond a single algorithm. The XOR equation is the smallest part of the system. The consequential work lies in preserving local repair, making remote reads tolerable, constraining recovery, and moving the installed data safely. At exabyte scale, redundancy is not a static format. It is an operating model.
Source and copyright notice
This article is an independent editorial digest of the publicly available FAST 2026 paper. The design, measurements, assumptions, and limitations were restated in new language after checking the official USENIX paper page. No sentence, table, Apple logo, facility image, or publisher figure is reproduced. The explanatory figure and thumbnail were created for this article; the hardware field is a generic editorial material plate with no company marks. Copyright © 2026 the authors.