An edge sensor and an AI cluster can both claim better energy efficiency while measuring different work, accuracy, and system boundaries. MLPerf Power addresses that problem with one methodology spanning microwatts to megawatts, built by participants from more than twenty organizations[1].
Define the boundary before comparing the number
The framework synchronizes useful-work and power measurement, specifies idle treatment and measurement equipment, and inherits MLPerf’s correctness thresholds. Sixty systems yielded 1,841 results that passed the reproducibility rules. Its central contribution is procedural: energy must be reported with the workload, scenario, quality target, and components included in the meter.
Results show why that discipline matters. In the reported BERT submissions, moving from 99% to 99.9% accuracy made inference about half as energy-efficient on average, although some systems resisted the loss. Diffusion image generation consumed 1,082× more energy per inference than the paper’s smallest compared workload. Data movement also grows into a first-order term at system scale and can be missed by device-only power readings.
What we take from it
MLPerf Power does not produce one universal winner. It makes the denominator auditable. Buyers should ask whether a result includes accelerators, host CPUs, networking, cooling, and idle periods, and whether quality targets match. Architects should use the same rules longitudinally so hardware, software, quantization, and scheduling improvements can be separated rather than collapsed into marketing efficiency.
Source and attribution
This article is an independently written digest with an original figure. It reproduces no paper prose, chart, or table. The public preprint is available on arXiv; the authoritative proceedings version is © IEEE 2025.