Every HBM system pays the same toll. The DRAM stack sits millimeters from the compute die, the two talk across an interposer through a PHY on each side, and moving one bit into an HBM4 base die costs roughly 2.4 pJ before any computation happens. The toll is not incidental: in token-by-token generation, where each output token requires streaming the active weights and KV state past the compute, memory traffic is the workload, and the interface energy scales with every byte of it. At Hot Chips 2026, d-Matrix presented early silicon for an accelerator named Raptor that removes the toll booth instead of widening the road[1]. A TSMC 4 nm compute die is bonded face to face onto a custom DRAM die at a 36 µm microbump pitch, the PHY disappears, and the vertical interface moves bits at 0.37 pJ, about 6.5× less energy than the HBM4 path. The advertised result is 100 TB/s of memory bandwidth from a single card. The number that requires explaining is the other one: that card carries 32 GB.

Coming three weeks after the HBF specification promised 512 GB stacks at HBM-class bandwidth, Raptor makes an instructive mirror image. The memory wall is being attacked from both ends at once: HBF trades latency for capacity, and Raptor trades capacity for bandwidth. We summarize the architecture, the packaging choices that make it work, and the scope of what has actually been demonstrated.

From SRAM to stacked DRAM

Raptor is the second act of an architecture d-Matrix has been building for years. Its shipping accelerator, Corsair, implements digital in-memory computing (DIMC): multiply-accumulate logic lives inside the memory arrays, so weights stay put and the compute comes to them. A Corsair chiplet packs 256 DIMC cores alongside 256 MB of SRAM[5], which delivers enormous bandwidth per byte but rations capacity in megabytes; models must shard across many chiplets, and SRAM’s cost per bit sets a hard ceiling on how far that scales. The obvious question was whether the same memory-side computing discipline could sit on a medium with DRAM’s density.

3DIMC is the answer, and it arrives with unusual provenance for a startup: a custom DRAM die co-developed with Alchip, announced in November 2025[3], and a dedicated test chip, Pavehawk, that validated the stacked DRAM-plus-logic combination in the lab before any product claim was made[4]. What Hot Chips 2026 added is the first look at Raptor itself, the commercial design that stacks the two dies in volume.

The stack is upside down

The packaging inverts the arrangement every HBM system uses, and each inversion has a reason. Logic goes on top: a liquid-cooling cold plate presses directly on the 4 nm compute die, instead of asking heat to escape through a thermally sensitive DRAM stack. DRAM goes underneath and moonlights as the interposer: PCIe and die-to-die signals travel down through the DRAM die’s TSVs to the package substrate, so the memory die is simultaneously storage and wiring[1][6]. Between the two dies there is no SerDes, no PHY and no interposer traversal, only a face-to-face microbump field at 36 µm pitch. Distance is what a PHY exists to overcome; remove the distance and the PHY, with its power and its latency, goes with it.

The claimed consequences compound. Per bit, the vertical interface costs 0.37 pJ against roughly 2.4 pJ for HBM4’s path. Per card, aggregate memory bandwidth reaches about 100 TB/s, a figure HBM systems reach only by summing eight or more stacks on the largest GPUs, and d-Matrix positions it as roughly 10× the effective inference bandwidth of HBM4-based platforms. Against NVIDIA’s Rubin generation, the company claims about 20× higher bandwidth density and 13.5× better power efficiency for the memory subsystem[6]. Note that all of these are vendor projections anchored to early silicon, not independent measurements of a shipping product.

An HBM system and Raptor, side by side. a, The conventional arrangement places DRAM stacks beside the compute die and pays a PHY and an interposer crossing on every access, at roughly 2.4 pJ per bit into an HBM4 base die. b, Raptor bonds the 4 nm compute die face to face onto a custom DRAM die at a 36 µm pitch; the cold plate contacts logic directly, the DRAM doubles as the interposer through its TSVs, and the vertical interface moves bits at 0.37 pJ. Original figure created for this article.

What 32 GB buys, and what it costs

A 32 GB card cannot hold a frontier model, and d-Matrix does not pretend otherwise. The deployment story is horizontal: the company describes a 72-card scale-up domain hosting a frontier-class model (its example is Kimi K3 at a 1M-token context), with each chiplet contributing about 1 TB/s of die-to-die bandwidth to hold the shards together[1]. The bet is that for latency-bound, small-batch token generation, the economics of many small, extremely fast memories beat the economics of few large, merely fast ones. This is precisely the opposite bet from HBF, which assumes the valuable working set is huge and read-mostly. Both can be right, because they target different phases of the same serving stack: decode wants bandwidth per active byte, and prefill plus weight storage wants capacity.

The honest caveats mirror the honest appeal. Capacity per card is 16× below an HBM4 GPU class device, so utilization depends on sharding software and on the interconnect between cards, the layer where, as our coverage of scale-up fabrics argued, determinism rather than raw bandwidth decides behavior at scale. Custom DRAM from a startup and a design-services partner enters a supply chain that HBM incumbents have spent a decade industrializing. Thermal headroom for the DRAM under a hot logic die is managed, not eliminated, by the inverted stack. And no shipping timeline has been announced; what exists today is validated test silicon and an early product die.

The memory wall, attacked from both ends. a, On a capacity-versus-bandwidth map, HBF extends the hierarchy rightward (512 GB per stack at HBM-class bandwidth) while Raptor extends it upward (100 TB/s per card at 32 GB); HBM4 sits between them. b, The stated division of serving labor: bandwidth-bound decode favors small, extremely fast memory close to compute, while weight storage and prefill reuse favor large, read-mostly tiers. Original figure created for this article.

The claims, sorted by evidence

It is worth separating what Hot Chips 2026 established from what it projected. Established: the 3D bond works, Pavehawk silicon has run in the lab, and a compute die on a custom DRAM die with no PHY between them is a manufacturable object at TSMC 4 nm. Measured on that silicon: the 0.37 pJ/bit vertical interface. Projected onto the future product: the 100 TB/s per card, the 10× inference advantage over HBM4 platforms, and the 20× bandwidth density and 13.5× power efficiency against Rubin, which compare an unshipped card to an unshipped platform. This ordering is not a criticism; it is exactly how Panmnesia’s CXL switch paper split its evidence between measured silicon and calibrated projection, and we apply the same reading discipline here.

What is demonstrated and what is projected. a, Demonstrated: face-to-face bonding at 36 µm pitch, Pavehawk test silicon validated in the lab, and a measured 0.37 pJ/bit vertical interface. b, Projected for the product: 100 TB/s per 32 GB card, roughly 10× effective inference bandwidth versus HBM4 platforms, and 20× bandwidth density with 13.5× power efficiency versus Rubin, all vendor figures for unshipped hardware. Original figure created for this article.

What we take from it

We believe the durable insight here is architectural rather than competitive. For two decades the memory interface has been the negotiated border between two industries, and every negotiation produced a PHY. Raptor is the clearest demonstration yet that when one company controls both dies, the border can simply be abolished, and the 6.5× energy gap between 0.37 and 2.4 pJ/bit measures what the border was costing. Whether d-Matrix converts that into a business depends on software, supply chain and the interconnect questions above. However, the direction of travel is hard to unsee, and the incumbents have noticed: memory vendors are already moving logic into base dies and studying processing near memory. HBF widens the hierarchy’s floor; Raptor raises its ceiling. The AI memory system of the early 2030s increasingly looks like both at once, with HBM squeezed into the role of the middle tier, and the interesting engineering, as usual, in the software that decides which byte lives where.

Source and attribution

This article is an editorial summary prepared for Silicon and Systems. It restates, in our own words, material from d-Matrix’s Hot Chips 2026 presentation of Raptor and the company’s public technical blogs and announcements, together with the independent conference coverage cited above. No text, figures or slides from those sources are reproduced here, and the figures on this page were created for this summary. The presentation and company materials are (c) d-Matrix, Inc. 2026.