Cloud-native relational databases separate compute from shared storage so read-only nodes and capacity can scale independently. Compression should reduce the shared layer’s dominant flash cost, but the database exposes several incompatible I/O classes. A redo record sits on transaction commit latency. A page read sits on query latency. A background page write can tolerate more work. Applying one codec and one block size to all three wastes either capacity or time.
Software compression offers algorithm choice and database context, yet consumes host CPU and needs an index from logical offsets to variable-size payloads. Computational storage performs compression beside flash and hides mapping behind a block interface, but fixed hardware cannot adapt easily to data or workload changes. PolarStore places both in series and then removes compression from the paths where its marginal benefit is smaller than its latency.
The deployment scale matters. Alibaba reports thousands of PolarDB storage servers, more than 100PB of managed data, 6,000 first-generation and 14,400 second-generation computational storage devices. The paper is as much a report on making compression hardware operable as it is a codec design.
Two layers divide granularity and flexibility
PolarCSD compresses data inside the storage device. Its logical address space is larger than physical flash, and an internal translation layer maps 4KB logical blocks into compressed byte ranges. Hardware offload keeps compression computation and variable-length placement away from database CPUs. The device is provisioned for an expected ratio rather than the worst possible logical capacity.
The software layer operates on 16KB database pages. It can choose a stronger codec for compressible, colder pages and a lower-latency codec for hot pages. Its result then enters the hardware layer, which can capture redundancy left by software alignment, metadata, and uncompressed I/O. The layers are not expected to multiply their ratios independently; the measured end-to-end ratio is the procurement value.
Variable-length hardware data requires byte-granular indexing and crash-consistent mapping. PolarCSD 1.0 ran its flash translation work on dedicated embedded resources but suffered CPU and memory contention at scale. PolarCSD 2.0 moved management threads to the host and changed hardware resources and interfaces. Host execution appears less isolated, yet stronger processors, explicit scheduling, and fewer constrained embedded resources removed slow I/O events attributed to device-side contention.
Redo bytes are valuable for durability, not compression
Redo writes are small, frequently recycled, and synchronous with transaction commit. Compressing them in the software layer produced little persistent capacity benefit but delayed every replicated durability operation. In ablation, adding Zstd software compression on top of hardware improved ratios by 21.7% to 50.3%, yet reduced throughput 19.6% versus hardware-only storage. Average three-replica redo latency rose from 59 to 79 microseconds.
PolarStore bypasses both compression layers for redo and places it on an Intel Optane performance tier. User pages remain compressed. This change reduced the throughput gap to 8.9% relative to hardware-only compression without materially changing the user-data ratio. It is a database-specific decision that a generic block compressor cannot infer.
The bypass illustrates the correct objective. A byte retained for minutes should not consume the same optimization budget as a page retained for years. Cost models should multiply saved physical bytes by residence time and divide by added latency on the affected operation. Redo can have a high write rate but a small steady-state footprint.
Codec selection compares saved I/O with decompression time
Zstd usually produces smaller pages but requires more compute than LZ4. On a compressed computational device, the smaller output can also reduce storage read time enough to offset slower decompression. PolarStore estimates both page sizes rounded to the 4KB device unit, measures decompression latency, and selects Zstd when saved bytes per added microsecond exceed 300B/µs. If host CPU utilization exceeds 20% or more than 30% of a page has changed, it selects LZ4 directly.
The choice is recalculated in a background write path. In the reported worst-case experiment every update triggered selection, increasing page-write latency, but foreground operations did not wait for it. Compared with Zstd-only software compression, adaptive selection reduced average page-read latency by about nine microseconds and brought database throughput to within 2.1% of the uncompressed P5510 baseline.
Capacity loss from faster pages was small. Across four production datasets, selection used 0.7% to 2.6% more space than always choosing Zstd. Codec distribution varied materially across finance, food-and-beverage, wiki, air, and transport data, supporting a per-page rather than cluster-wide decision.
A page log converts tail reads into one access
Read-only database nodes replay redo to construct pages newer than their local checkpoint. If required log records have fallen out of the cache, a page build can issue many scattered storage reads. Compression can lengthen each access and worsen the tail. PolarStore maintains a per-page log so the required records are retrieved together.
The evaluation delayed a read-only node about one second behind the read-write node to sustain log-cache pressure. With fewer than 128 reader threads, the page log reduced p95 latency by 28.9% to 39.5%. Above 128 threads, the read-only node became CPU-bound and software queueing dominated, so fewer I/Os no longer controlled the result.
This boundary is important. An I/O optimization cannot repair compute saturation. Operators need page-log hit rate, scattered reads avoided, page reconstruction CPU, and run-queue delay. Enabling the mechanism at already CPU-bound nodes can add metadata without improving queries.
Production cost must include the device premium
The first-generation compressed cluster reached a 2.35 ratio. Its PolarCSD hardware cost 1.45 times the normalized P4510 comparison, so effective cost per logical GB was 0.62 rather than 1.00. It also ran about 10% slower and disabled software compression and two database optimizations to avoid contention.
The second generation lowered relative device cost from 1.45 to 1.32 and enabled the full dual-layer stack. Its production ratio was 3.55, placing normalized cost per logical GB at 0.37 versus 0.91 for the contemporary P5510 cluster. The paper describes this as approximately 60% storage-cost reduction. Sysbench on a 480GB database, eight storage nodes, an eight-core 32GB compute instance, and a 16-thread client found performance parity between second-generation compressed and uncompressed clusters.
The comparison is not a claim that every dataset compresses 3.55 times. Hardware-only ratios across four datasets ranged from 2.12 to 3.84. Logical capacity also becomes uneven when tenants compress differently. PolarStore schedules by both logical utilization and measured ratio, moving low-ratio chunks away from physically pressured nodes. In the cited balancing study, production clusters operated from 2.2-fold through 2.7-fold compression, with 87.7% of nodes inside the target balance zone.
Compression hardware becomes a cluster-management problem
Overprovisioning logical address space assumes a ratio. A tenant storing encrypted, precompressed, or high-entropy data can break it. Admission control must reserve physical capacity using conservative per-dataset estimates, and the control plane needs to react before a node’s real flash fills. A logical-free-space metric alone is unsafe.
The index and translation layer are durability-critical metadata. Recovery must prove that a committed logical page resolves to exactly one compressed physical range after power failure. Firmware update, host translation service restart, and partial replica failure need fault injection. A compression ratio is not useful if its mapping cannot be rebuilt within the database’s recovery objective.
Tail behavior also needs monitoring by codec, ratio bucket, operation class, and device generation. The production data reported only 7.91×10^-7 reads and 1.05×10^-6 writes above four milliseconds for the second-generation device distribution, substantially improving the first generation. Such rare events are still visible at millions of IOPS, so counts and workload impact matter more than a low percentage alone.
The design rule is to compress residence, not urgency
PolarStore’s strongest contribution is its separation of criticality. Long-lived pages receive hardware compression and optional software compression. Commit-critical redo bypasses it. Page reads choose a codec using saved I/O versus decompression time. Recovery-tail pressure receives a layout designed around a database page. Cluster placement corrects for tenant-specific ratios.
The 3.55 ratio and 0.37 normalized logical-GB cost show that specialized devices can pay back at 100PB scale. The first-generation result also shows that offload is not automatically isolation; constrained compute and memory inside a device can create tails that a host CPU avoids. Buyers should request a full ratio distribution, logical cost including device premium, database throughput, p95 and rare slow I/O, recovery time, and behavior for incompressible tenants.
Compression is therefore a placement problem across time as well as layers. The system should spend cycles where bytes live long enough to repay them and step aside where an extra microsecond delays a commit. PolarStore reaches performance parity only after making those exceptions explicit.
Source and copyright notice
This article is an editorial analysis by Silicon & Systems. It restates the architecture, production measurements, and limitations in our own words. No source sentence, table, or figure is reproduced; the figure was created for this article. The paper is available from the USENIX FAST 2026 presentation page. Copyright remains with the authors, 2026.