A NIC receive ring looks like one queue but implements two producer-consumer relationships. Software places empty buffers into descriptors for the NIC to consume. After a packet arrives, the NIC returns a full buffer for software to consume. Binding those directions to one circular order means an empty buffer cannot be reused until the NIC and CPU advance through the entire ring.
The default ring on many 100 Gb/s adapters has 1,024 entries so it can absorb bursts. With 1,500-byte buffers, each core reserves roughly 1.5 MiB of I/O working set. Across many cores this exceeds the last-level cache and the portion available to DDIO. New DMA writes evict unprocessed packets, the CPU fetches them from memory, bandwidth rises, and a slow core falls further behind[1].
Smaller rings fit the cache but drop larger bursts. Sharing one ring across cores reduces buffers but couples their delivery progress. An overloaded core can fill the common ring and block packets destined for idle cores. RxBisect starts from the claim that buffer capacity should be shared while packet delivery remains independent.
Allocation and reception become separate rings
Each core exposes an allocation (Ax) ring containing empty buffers. A bisected reception (Bx) ring carries notifications for packets delivered to a processing core. One Bx ring can draw from several Ax rings. If its preferred allocator is empty during a burst, the NIC takes a buffer from another associated Ax ring while still notifying the originally selected Bx consumer.
Receive-side scaling therefore continues to choose the processing core. Only buffer ownership moves. The union of empty buffers becomes a shared resource without a lock on one shared packet queue. Software can use small per-core Ax rings or dedicate a few allocation cores while keeping larger Bx rings for notification bursts, since Bx descriptors need not point to a permanently reserved packet-buffer set.
The hardware pays another descriptor operation when the chosen buffer comes from a different allocation ring, but packet data still lands in the same DDIO-local memory. The paper argues that the dependent DMA critical path matches existing private and shared designs. Allocator traffic was below 0.2% of cycles in the evaluated imbalance extremes.
Cache capacity is the real sizing constraint
On a 16-core processor with a 22 MiB LLC and two 100 Gb/s ConnectX-5 NICs, larger private rings cut measured throughput by as much as 20%, inflated latency as much as 37-fold, and raised memory-bandwidth demand as much as 4.9-fold. Line rate held when the working set fit the DDIO ways at ring sizes no larger than 128. The result then declined as buffers exceeded DDIO capacity and again after exceeding the full LLC.
Using fewer cores is not a general answer. Large packets reached peak throughput before all 16 cores, but 64-byte packets needed every core. A system must provision processing parallelism and burst absorption for changing traffic without permanently multiplying buffers by both maximum core count and maximum ring depth.
RxBisect and a shared-ring design matched a 1,024-entry private ring’s no-drop throughput with one eighth of its buffer working set. Unlike the shared ring, rxBisect does not make the slow consumer own the buffer pool. In a synthetic skew, shared-ring throughput fell as much as 60% when one target core processed packets slowly, while rxBisect retained line rate until its software emulator became the limit.
Real traces test the imbalance claim
The evaluation used NAT and load-balancer network functions plus the MICA key-value store. Servers carried two 16-core Xeon Silver 4216 processors, 22 MiB LLCs, and back-to-back pairs of 100 Gb/s ConnectX-5 NICs. RxBisect was emulated in software because commodity NICs do not expose the proposed interface.
With an imbalanced CAIDA trace and co-located PageRank, rxBisect exceeded an idealized dynamic shared-ring policy by 16% for load balancing and 20% for NAT. Adding STREAM memory pressure preserved an emulated advantage up to 16%. At low traffic, shared-ring synchronization spent up to 34% and 46% more cycles per packet than private rings in the two functions, while rxBisect avoided that shared-tail lock.
Against ordinary private rings, reported throughput improved by up to 37% and latency by up to 11 times when rxBisect sustained line rate and the baseline queued packets. Those large latency ratios describe overload transitions, not a constant per-packet acceleration.
Emulation makes the result conditional
The authors place the emulator on another NUMA node, keep packet buffers and workers local to the NIC, and reproduce doorbells and an extra Bx DMA. For existing ring schemes, emulation reduced throughput by up to 12% and increased latency by up to 94%, suggesting a conservative comparison. Still, a real NIC may expose pipeline, cache-coherence, descriptor-fetch, and firmware constraints that software does not reproduce.
Native adoption changes the ABI between drivers, DPDK, firmware, and silicon. The NIC must select among nonempty Ax rings, report which buffer it consumed, preserve isolation, and recover cleanly across resets. Queue configuration also needs limits so one tenant cannot lend memory outside its protection domain.
Descriptor accounting becomes a correctness surface once allocation and delivery advance independently. Software must know whether a returned buffer belongs to its local allocator or must be released through another core’s cache. Meanwhile, the NIC must never consume an entry twice as Ax and Bx heads wrap at different rates. The interface also needs a defined response when every associated Ax ring is empty. Dropping, pausing, or redirecting a packet changes burst behavior and observability, so counters must distinguish receive congestion from allocation starvation.
NUMA placement can reverse the cache benefit. Sharing buffers among cores on one socket preserves the intended LLC locality, but borrowing from an Ax ring on another socket may replace cache pressure with interconnect traffic and remote-memory latency. A production policy should constrain association sets by NUMA domain and expand them only after local capacity is exhausted. That hierarchy preserves common-case locality while retaining an explicit emergency pool for unusually skewed bursts.
The interface complements rather than replaces load balancing. RSS or application-aware steering still decides which core should process a flow. RxBisect prevents that core’s temporary buffer shortage from withholding spare capacity elsewhere. It also complements LLC partitioning and header-only DDIO schemes because those reduce interference while the split interface reduces the buffer set itself.
The reusable lesson is to separate coupled resources
The conventional ring bundled burst capacity, memory ownership, and processing order. That bundle was reasonable when one queue served one core at lower line rates. At hundreds of gigabits, provisioning it per core turns descriptor depth into a cache-capacity tax. Sharing the whole queue removes the tax but spreads a slow consumer’s backpressure.
RxBisect shares only the fungible resource, empty packet buffers. It keeps the scheduling resource, the delivery queue, independent. The same question applies to storage submission queues and accelerator command rings: if one circular structure carries capacity in one direction and completed work in the other, separating those roles may improve both locality and imbalance tolerance.
Deployment decisions should measure working-set bytes per core, DDIO ways, burst-loss tolerance, flow skew, allocator crossings, memory bandwidth, and tail latency near saturation. The proposed numbers justify hardware exploration, not immediate replacement. A native prototype must show that the extra selection and notification logic preserves line rate, isolation, reset semantics, and driver simplicity.
The prototype also needs mixed packet sizes and changing flow affinity over long intervals. Average balanced traffic is the easiest case for every receive interface; rxBisect matters when processing time and arrival rate diverge across cores. Hardware counters for Ax depletion, cross-core buffer borrowing, Bx occupancy, DDIO misses, and allocator handoffs would make the claimed resilience diagnosable rather than implicit. Operators could then determine whether a throughput loss came from compute saturation, exhausted buffer capacity, or an unfavorable NUMA association.
Source and copyright notice
This article is an editorial analysis by Silicon & Systems. It restates the interface, measurements, and limitations in our own words. No source sentence, table, or figure is reproduced; the figure was created for this article. The paper is available from the USENIX OSDI 2025 presentation page. Copyright remains with the authors, 2025.