Most AI accelerators expose a matrix multiplication shape and ask the compiler to decompose everything into it. FuriosaAI’s ISCA 2024 paper reverses that contract. Its Tensor Contraction Processor (TCP) starts from the more general operation that underlies matrix multiplication and many tensor transforms, then lets hardware regroup its compute resources around the current axes[1].
Flexible slices without tiny-core programming
TCP uses coarse processing elements so software does not have to schedule a sea of small cores. Each element can act as one large unit or split into independent slices. A circuit-switched fetch network connects those slices to data sources, allowing an operand to be reused across several compute units. Broadcast and local input buffers add reuse inside the contraction engine. The compiler searches tensor shapes, loop order, partitioning, and the available slice topology together.
This middle ground matters. One rigid array is easy to program but wastes lanes on mismatched shapes. Hundreds of tiny cores adapt better but move scheduling and network complexity into software. TCP keeps a coarse programming surface while allowing the physical engine to fragment only when the operation benefits.
A memory-heavy 150 W design
The 5 nm chip integrates 256 MB of SRAM with 384 TB/s of internal bandwidth and 48 GB of HBM3 delivering 1.5 TB/s. Peak arithmetic is reported as 256 TOPS for BF16, 512 TOPS for FP8 or INT8, and 1,024 TOPS for INT4, within a 150 W thermal design power. Those numbers describe ceilings; the architecture’s claim is that flexible contraction mapping keeps a larger fraction useful across prefill and decode shapes.
For Llama-2 7B, the paper reports 2.7× the performance per watt of H100 and 4.1× that of L40S over the tested configurations. The authors use public NVIDIA results and note computational equivalence between Llama and Llama-2 7B in the compared path. The result is therefore evidence for the selected sequence lengths, batch sizes, precisions, and power assumptions, not a blanket ranking across models.
What we take from it
TCP’s strongest idea is not a peak TOPS number. It is that the abstraction visible to the compiler should match the algebra and should remain divisible at runtime. If this software stack continues to cover new operators, a coarse but shape-flexible engine can offer a credible alternative to both rigid systolic arrays and general GPUs. The remaining test is deployment breadth: compiler maturity and stable utilization across rapidly changing models will decide whether the silicon’s flexibility survives outside the case study.
Source and attribution
This article is an editorial summary prepared in our own words. It reproduces no paper text, figures, or tables. The explanatory figure was created for this article. The authoritative proceedings version is © IEEE 2024.