Tape is cheap partly because most of it is inactive. Huawei Cloud’s production library configuration stores 1,000 cartridges behind four drives[1]. A cartridge consumes no drive and no media power while it sits on a shelf, but it also cannot serve data. The robotic exchange, unloading the previous cartridge and loading the next one, takes about 80 seconds on the reported platform. Each drive then supplies 360 MB/s. The system owns more than 10 PB of capacity in one library, but only four pieces of media can be active at once.
TapeOBS does not try to hide this asymmetry with a faster queue. It changes the timing contract. User writes first become durable in an HDD staging pool, then move to tape in bulk. A restore request schedules movement back to disk before the object can be read. The service offers restoration in hours, giving the control plane enough time to group requests, reduce cartridge exchanges, and stream each mounted tape. The durable contribution is therefore not a new tape format. It is the use of asynchrony as a system resource.
The design began a phased release late in 2022, started customer service in 2024, and held a raw corpus measured in the hundreds of petabytes by the paper’s cutoff. Production evidence matters here because a tape architecture is defined as much by robotics, firmware, and maintenance as by data structures. The paper reports 17 library-related incidents over roughly 1.25 years and fewer than 200 libraries, making failure handling part of the analysis rather than an appendix.
Capacity and bandwidth are deliberately decoupled
One reported library contains 1,000 cartridges with 10,742 GB per cartridge, for 10.24 PB of uncompressed capacity. Four drives provide an aggregate theoretical stream rate of 1.44 GB/s. This ratio is economically useful because cartridges cost less than drives and require no power while idle. It also means a random sequence of objects can spend more time exchanging and seeking media than transferring bytes.
The paper’s 10-year model begins with 100 PB and assumes 50% annual data growth. Under those assumptions, tape has 2.68 times lower capital expense and 16.11 times lower operating expense than the HDD design, producing a 4.95 times lower total cost. The authors also cite a 10-year tape life against five years for HDDs and a 44% floor-space saving in their environment.
These numbers are not a general price guarantee. They depend on growth, replacement schedules, facility cost, library utilization, migration policy, and restore demand. The model is still useful because it identifies why tape can win. Media are dense and passive, drives are shared across a large installed capacity, and refresh happens less often. Every attempt to make tape behave like an always-active disk spends against those advantages.
Four percent of disk capacity creates scheduling freedom
The production HDD pool is about 4% of tape capacity. For writes, it absorbs incoming objects under a highly available disk contract and returns control without waiting for a cartridge. The system later drains staged data into the tape pool. For reads, it materializes restored objects so ordinary object reads can proceed after the hours-level restore operation completes.
This small pool changes the queue from urgent requests into movable work. Write bursts can be absorbed at disk aggregate bandwidth. Restore requests can wait until enough work targets the same cartridge or nearby positions. Tape traffic can run at a steadier rate even when user traffic is irregular. In one 24-hour production interval, the HDD pool stayed between 71.625% and 71.675% utilized while the tape pool accepted data at a comparatively stable hourly rate.
TapeOBS uses a 75% disk-pool watermark. The remaining quarter covers customer bursts, disk-pool repair, and performance loss that would appear near full capacity. That headroom is also a failure budget. When an entire tape library is unavailable, incoming writes can continue to land on disk while operators repair the library. The measured daily ingest was below 4% of disk-pool capacity, so the reserved space could provide dozens of hours in the described condition.
The system implication is precise: staging capacity should be sized from recovery time and burst volume, not from the total archive. Four percent works for the reported traffic and maintenance window. A workload with a larger ingest burst or a shorter restore promise needs a different fraction. The relevant equation is usable headroom divided by net ingest rate during degraded operation.
Lifetime grouping turns append-only media into an advantage
Modern tape is effectively append-only at the software level. Reclaiming scattered dead objects would require copying live data elsewhere. TapeOBS groups staged objects by expected lifetime and appends similar-lifetime data together. When the retention period expires, a larger portion of a cartridge or erasure-coding group can be reclaimed without relocating unrelated long-lived objects.
This policy is difficult without persistent staging. The system cannot keep one cartridge mounted for every lifetime class because there are only four drives and mounting is expensive. Disk lets objects accumulate until the scheduler has enough data to make a coherent tape write. Thus, the HDD pool is not merely a faster cache. It is where the service converts fine-grained arrival order into a physical layout that fits sequential media.
The placement prediction can be wrong. Users can change retention, delete early, or keep data longer than expected. Operators should therefore measure the live-data fraction of reclaimed tapes and the bytes copied per byte freed. A high ingest rate with poor lifetime correlation can turn future garbage collection into the hidden price of the initial write.
Batched erasure coding protects objects without activating every drive
Production uses a 12+2 erasure-coding layout across fourteen tape racks, for 1.17 times storage redundancy. A conventional object-level stripe can spread every object across many tapes. Restoring a small object would then require several cartridges and drives. TapeOBS batches multiple objects into a larger coding unit so that an individual object often resides on fewer tapes even though the batch as a whole is protected across racks.
The benefit is lower drive demand and fewer exchanges during normal restoration. The cost appears during degradation. If a missing fragment must be reconstructed, the batch layout can require more surviving data to be read than a fine-grained object stripe. The authors accept this because degraded reads are rare compared with ordinary restores and because active drives are the constrained resource.
This is an important denominator choice. The coding rate alone does not describe archive efficiency. A design must count cartridges mounted per restored object, drive-minutes consumed, bytes read during repair, and the duration for which a failed rack reduces available restore bandwidth. Batched coding moves work from the common path into the failure path, which is rational only when the failure frequency and repair window justify it.
Metadata must not live behind the same robotic seek
Locating a small object by reading tape metadata would turn every lookup into a physical operation. Each library therefore has two NVMe SSDs that host a simple key-value store for persistent-log metadata and buffered data. The system can resolve a tape address without mounting or scanning the cartridge. When a tape fills, metadata are also written to a dedicated tape partition for recovery. Every 4 KB data block carries an integrity field so the medium can be reconstructed if the SSD metadata are lost.
The design separates the live index from the archival authority. SSDs provide fast operational state; tape preserves enough metadata and checksums to rebuild that state. This separation is required because mirrored SSDs can still fail together or become inconsistent with media after a software fault. A recovery drill should therefore rebuild a library index from tape and validate both the completion time and the resulting object map.
Metadata size remains manageable because TapeOBS indexes persistent-log units rather than every object at full detail inside the library. The object service and index layers retain higher-level mappings, while local metadata identify the sub-log and physical tape position. Choosing the recovery unit carefully prevents a petabyte archive from requiring a proportionally expensive always-on metadata service.
Stable streaming requires controlling the submitter
Tape drives select an internal streaming speed based partly on how quickly the host supplies data. In one trace, a drive initially averaged 335.94 MB/s, then fluctuated and settled near 168.65 MB/s for 285 seconds because irregular submissions led it to choose a lower rate. The library scheduler fixed the behavior by observing drive-buffer occupancy and rate-limiting host submissions to match the estimated media speed. Average bandwidth became 336.53 MB/s, with brief drops when the head reversed direction between wraps.
This result reverses a familiar storage instinct. Sending requests as quickly as possible can halve throughput when the device interprets a jittery queue as an inability to sustain streaming. The control objective is not maximum instantaneous submission. It is a stable pipeline that keeps the tape moving at the appropriate speed.
TapeOBS also dedicates two drives to writes, one to restores, and one to internal work in each four-drive library. Static allocation sacrifices flexibility but prevents a repair or restore from repeatedly displacing the write cartridge. Since production traffic is overwhelmingly write-heavy, the chosen split matches the observed workload. A service with frequent restores would need dynamic reassignment or a different library ratio.
Production traffic validates the asynchronous premise
Across the five largest customers, writes account for at least 99.325224% of object operations. The highest read share is 0.674776%; two of the customers issued no reads in the observed period. Objects below 500 MB occupy 93.81% of total space, with the 50 to 100 MB range alone representing 69.95%. TapeOBS is therefore not relying on a generic belief that archive data are cold. Its deployment sees a workload in which restoration is rare enough to schedule and objects are small enough that naive striping could waste many mounts.
The tape pool averaged 118.81 thousand coded-stripe writes per minute during one day. Each reported write operation represents 7 MB after 12 data and two parity fragments, producing 831.67 GB per minute at that measurement point. The authors explicitly distinguish this instrumentation unit from an object call. That distinction prevents a tempting but incorrect conversion into user-object throughput.
Writes complete at the SSD-resident staging layer of each tape rack, not after magnetic media movement. Median latency is 18.51 ms and the 99th percentile is 27.75 ms for a 7 MB coded stripe. Roughly 10 ms comes from the kernel TCP/IP network path, 1 to 4 ms from SSD data and metadata writes, and the remainder from parity, checksums, copying, and service logic. These are staging-write latencies. Durability on tape follows asynchronously.
Failure records show why the disk buffer is operational infrastructure
The deployment recorded 17 tape-library incidents. Four were drive-software defects, four were drive failures, four involved drives not recognizing cartridges, one made a drive disappear, two involved a stuck robotic accessor, and two disconnected a head server from its library. The sample is too small for a general hardware failure rate, but it identifies distinct operational consequences.
A failed write drive reduces ingest capacity. A failed read drive converts restores into degraded reads from other racks. Failure of the internal-work drive delays maintenance. A stuck robot or disconnected library removes the entire unit, at which point the HDD pool keeps accepting writes and erasure coding supplies reads from surviving racks. Dedicated roles make the performance impact predictable but can leave spare capability unusable unless operations can reassign drives safely.
An archive service should expose this degraded state to its control plane. It needs estimates for remaining disk-buffer hours, restore capacity after each rack loss, repair traffic, and the number of cartridges whose only readable copy depends on a particular drive generation. Those values turn a mechanical fault into a scheduled maintenance decision rather than a surprise capacity crisis.
Tape is appropriate only when delay has business value
TapeOBS demonstrates a strong fit between an hours-level restoration contract and a medium that rewards batching. It does not show that tape should sit behind a normal low-latency object API. If applications require immediate reads, frequent overwrites, or unpredictable deletions, the scheduler loses the freedom that produces the cost advantage. An HDD or flash tier may be cheaper once the drives, staging capacity, and operational complexity needed to meet the tighter contract are included.
The adoption test begins with four workload measurements: ingest rate, restore frequency, retention correlation, and maximum acceptable restoration time. Operators can then calculate staging headroom, drive demand, mount frequency, and expected garbage collection. The paper’s 4.95 times TCO result belongs to its ten-year growth model. The transferable result is that an archive service should sell time as part of the interface and spend that time to align requests with the physics of the medium.
Source and copyright notice
This article is an independent Silicon & Systems editorial digest of the authors’ public FAST paper. It restates mechanisms, deployment observations, measurements, and limitations in original language. No paper sentence, table, or publisher figure is reproduced. The figure and thumbnail were created for this article from reported facts and a generic server material plate. Copyright for the source paper remains with its authors. The complete paper is available through the official USENIX FAST 2026 page.