The AI silicon genre has a house style: die shot, peak TOPS, a benchmark table, and a wave toward future work. Meta’s ISCA 2025 industry paper on MTIA 2i, the second-generation chip now serving ranking and recommendation inference to billions of users, opens by naming what that genre leaves out[1]. Architecture, the authors argue, is the smaller half of making an in-house chip pay; the larger half is co-designing models to fit the silicon, proving the chip flexible enough to survive model evolution, and absorbing everything production throws at hardware that no vendor stands behind but yourself. The headline number is a 44% average reduction in total cost of ownership against GPUs for the models launched so far, and the paper is largely an itemized account of what that number cost to earn.
We summarize the argument in our own words below.
A memory hierarchy with a hole where HBM should be
MTIA 2i is a TSMC 5 nm die of 64 processing elements at 1.35 GHz, each pairing two RISC-V cores with fixed-function engines for matrix multiply, reduction, SIMD work and layout transforms, delivering 354 INT8 TOPS (708 with structured sparsity) in an 85 W envelope, roughly 3× its 2023 predecessor[2] on compute, network-on-chip bandwidth and SRAM bandwidth. The unconventional part is what feeds it. There is no HBM: 256 MB of on-die SRAM at 2.7 TB/s sits over 64 to 128 GB of LPDDR5 at 204.8 GB/s, a 13× bandwidth cliff between tiers. The bet is that recommendation models, unlike large Transformers, have working sets with enough locality to live in SRAM: production results bear it out, with dense layers hitting over 95% SRAM residency and even the irregular embedding lookups keeping 40 to 60% of accesses on-die. Software chooses how the SRAM splits between a hardware-managed cache and software-pinned scratch, and an autotuning framework (kernel variants picked by nearest-neighbor search over a performance database, 1000× faster than exhaustive sweeps; batch sizes chosen by traffic replay) makes that choice per model, since Meta has far too many models to hand-tune. A deliberately small chip is the other half of the thesis: 24 of them share one Grand Teton server whose aggregate matches an 8-GPU box, and small units let a fleet with spiky, bursty demand allocate capacity in fine grains instead of idling monsters.
What the datasheet never tells you
The productionization chapters are the paper’s real contribution, and each reads like a decision a vendor datasheet would have made for you. LPDDR has no native ECC, and at design time Meta genuinely did not know whether inference could shrug off bit flips, so the question went to the fleet: among the first 1,700 servers, 24% showed memory errors, and injection experiments found that flips in embedding indices or the wrong bits of a weight reliably produced NaNs or corrupted rankings. Product teams vetoed absorbing that noise, so controller-side ECC went on, at a 10 to 15% throughput penalty that every number in the paper already includes. Overclocking ran the other direction: a 3,000-chip study across ten test suites showed the silicon held margin well past its 1.1 GHz design point, and the fleet now runs at 1.35 GHz, worth 5 to 20% end-to-end. Six months of production power data let the team cut the provisioned rack budget by nearly 40%, on the observation that 24 small chips rarely peak together. And owning the firmware turned silicon errata from crises into patches: a PCIe-ordering deadlock that surfaced on 0.1% of loaded production servers was mitigated by relocating one buffer from host memory to SRAM, shipped fleet-wide through a pipeline that released 23 firmware bundles in 2024, against the one or two a year the same team manages for third-party GPUs.
Model-chip co-design gets equal candor, including about features that failed. Dynamic INT8 quantization works (row-wise activations over static INT8 weights match FP16 quality) but the quantize-dequantize overhead cuts the DPE’s 2× advantage to about 1.6× on big layers, so FP16 still rules production. The 2:4 sparsity path stays mostly idle because the layers with enough sparsity to exploit are not the layers that decide model quality. What did pay was locality engineering: operator fusions that pin activation buffers in scratch SRAM, broadcast reads that stop 64 PEs from fighting over weight traffic (over 95% of DRAM bandwidth achieved on the worst shapes), and a hard-won organizational habit of rejecting model changes that break SRAM residency and offering ML engineers an equivalent that does not. The eight-month case study tells that story end to end: one of Meta’s top-five revenue models arrived at 50% of GPU cost-efficiency, grew 6.7× in complexity while being ported, and launched at 180%.
What the numbers say
Across nine production models spanning 15 to 1,000 MFLOPS per sample, MTIA 2i lands between 0.9× and 2.1× GPU cost-efficiency, averaging the headline 44% TCO reduction, with performance per watt clustered near parity (0.8 to 1.2×), which the authors attribute to GPUs’ decades of power tuning. Efficiency tracks SRAM fit rather than complexity per se: the winners are models whose activations pin in scratch memory at large batch, and the laggards are those that spill. Validation is unusually strong for the genre, since the serving stack can split live traffic between MTIA and GPU fleets running transformations of the same trained model, and the comparison is scored on advertising revenue and prediction quality, not just latency. The boundaries are stated with the same bluntness: Llama-class LLMs do not fit (an 8 B model meets prefill targets but the LPDDR-bound decode misses its 60 ms token budget), models beyond roughly 2 GFLOPS per sample outrun the SRAM strategy, and the software ecosystem still trails the CUDA world for the long tail of small models that cannot justify manual attention.
What we take from it
Set beside the other industry silicon we have covered, this paper completes a pattern. Vistara is Meta shipping unglamorous memory expansion; MTIA 2i is Meta shipping unglamorous compute, and both papers spend their prestige on operational truth rather than peak numbers. The chip’s design thesis also reads as a mirror image of the demand-side documents: where DeepSeek’s wishlist[6] begs for more interconnect and HBM for token generation, Meta’s recommendation fleet demonstrates a large, lucrative AI workload for which the correct answer was less memory bandwidth, bought cheaper, fronted by SRAM. We believe the durable lesson is the 44% itself being an average over survivorship: MTIA 2i wins because Meta reserved the right not to run every model on it, keeping GPUs for what falls outside the design point. In-house silicon does not need to beat the general-purpose incumbent everywhere; it needs a workload census accurate enough to know where it will, and the organizational machinery (autotuning, A/B against live revenue, weekly firmware) to hold that ground as models drift. That machinery, not the PE array, is what this paper actually documents, and it is the part competitors will find hardest to copy.
Source and attribution
This article is an editorial summary prepared for Silicon and Systems. It restates the argument of the paper cited below in our own words. No text, figures or tables from the paper are reproduced here, and the figures on this page were created for this summary. The paper appeared in the ISCA 2025 industry track and is published under a CC BY 4.0 license; (c) 2025 the authors.