The most consequential AI infrastructure papers published by industry after 2025 do not agree on one new accelerator, network, or scheduler. They agree on a boundary. The useful unit of optimization is no longer a GPU server. It is the path from a facility limit to completed model work, including every place where power, heat, bytes, queues, synchronization, and failures can interrupt that conversion.
This review examines nine primary papers released from January 2025 through August 28, 2026. The set spans Meta, Google, DeepSeek, Huawei Cloud, NVIDIA, Microsoft, Alibaba, and ByteDance. It is not a citation ranking or an exhaustive bibliography. We selected systems with an industry operator or deployment context, a disclosed mechanism, quantitative evidence, and a limitation that changes how the headline result should be used. The common result is more important than any single speedup: the datacenter is becoming a programmable runtime.
That statement has a precise meaning. A runtime observes state, predicts what will happen next, selects an action, and checks whether the action preserved a service objective. The 2025–2026 systems apply this loop to constraints that used to sit outside software: a 150 MW breaker envelope, future cooling capacity, compute-fabric bandwidth, storage placement, KV-cache locality, agent workflow choices, reinforcement-learning phases, GPU fragmentation, and recovery state. The loops operate from microseconds to decades, but their architecture is recognizably the same.
The first control plane is the building
Meta’s 2026 report on an approximately 83,000-GPU GB200 deployment starts six to twelve months before the hardware arrives. The facility offers a fixed 150 MW envelope across five buildings. A design based on the 1,200 W device limit would fit roughly 74,000 GPUs. Planning at 960 W admits about 86,000 in the model and approximately 83,000 after network and deployment constraints. Although each accelerator is slower, aggregate throughput is about 11% higher than the 1,200 W plan. Commissioning then raises the cap to 1,020 W after calibrated measurements reveal headroom. A coordinated runtime controller contributes another increment. The complete stack raises projected cluster throughput by about 14% without another megawatt of facility capacity[1].
The mechanism matters more than the selected wattage. Rack power-supply telemetry overestimated the load seen at upstream panels, while local main switchboards could remain tight even when the campus total had space. Meta therefore converts a static procurement budget into a hierarchy of measured, schedulable limits. Power Smoother reduces synchronized electrical valleys, and Dimmer adjusts ranks together using a seven-second average. If one rank alone were throttled, the other ranks would consume power while waiting for the new straggler. The actuator must understand the workload topology.
Google’s Prometheus study moves the same loop to a much longer horizon. It combines 25 years of historical weather observations with 20 years of climate projections across 30 datacenters. The reported design implication is an 11% average cooling-capacity increase and a 48% worst case. Prometheus then divides action into 20-year facility planning, two-year equipment decisions, and two-week operating forecasts[2]. The paper is not saying that software can route around a failed chiller forever. Moving a 10 MW workload can involve roughly 200,000 virtual machines and 3.2 PB of memory, and most evaluated sites could move less than 20% of load. Software provides a tactical actuator; physical cooling remains the capacity floor.
Read together, the two papers replace a familiar utilization question. The issue is not whether GPUs are busy. It is whether the next unit of electrical and thermal capacity can be delivered at the rack, at the right time, without violating the operating envelope. A fleet dashboard that omits panel hierarchy, transient behavior, weather exposure, and relocation cost is not a cluster dashboard. It is a partial device inventory.
Data motion becomes an explicit API
DeepSeek-V3 exposes the next boundary. The 671-billion-parameter model was trained on 2,048 H800 GPUs, whose NVLink bandwidth is 400 GB/s rather than the 900 GB/s available on H100. DeepSeek’s analysis shows that communication is not a tax paid after the model is designed. Expert parallelism, communication kernels, node placement, and hardware topology jointly determine which model architecture is economical[3]. DeepEP reports more than 40 GB/s per GPU across 16 to 128 GPUs, while parts of the runtime can reserve as many as 20 streaming multiprocessors for communication bookkeeping. The broader lesson is that FLOPS are sellable only after the system has scheduled the bytes that activate them.
Huawei Cloud’s AITURBO makes a similar change at the storage boundary. In one local datacenter, AI workloads accounted for over a tenth of the traffic delivered by cloud storage. The system adds a grouped I/O API so the storage layer can see a collective operation rather than thousands of unrelated file requests. It deduplicates the plan, stages data in host DRAM, and uses the compute fabric to broadcast data among accelerators. In a production training deployment, checkpoint writes are 3.9 to 58.8 times faster than SFSTURBO and as much as 5.9 times faster than Gemini. An inference integration reaches up to 1.28 times the throughput of Mooncake and cuts time to first token by 23% on a real trace[4].
Those maxima are not interchangeable benchmarks. AITURBO targets bulk transfers and reports coordination overhead reaching 45 ms at 64 XPUs. Small operations may not benefit, and sharing the compute fabric with training traffic creates a quality-of-service problem. The important result is the control surface. With only 286 lines added to Megatron for the evaluated training integration, the application expresses intent while the storage system decides how to move the group.
Strata applies the same principle to long-context serving. A single request can leave KV-cache blocks across GPU HBM, CPU memory, and SSD. Fragmented cache layouts turn a nominally large transfer into many small operations; loading can stall prefill; and a cache hit can still be a poor scheduling decision if the bytes arrive too late. Strata combines GPU-assisted I/O with cache-aware scheduling and a hierarchy spanning all three memory tiers. It is integrated into SGLang and has been deployed in production. The paper reports up to five times the performance of vLLM-LMCache and 3.75 times that of TensorRT-LLM, without harming the tested short-context workload[5].
The three systems elevate data location into runtime state. DeepSeek exposes topology to model execution. AITURBO exposes operation groups to storage. Strata exposes cache readiness to the request scheduler. None merely installs a faster device. Each gives software enough semantic information to answer a different question: which bytes will be consumed together, where are they now, and when must they arrive?
Schedulers must understand phases, not just resources
Agentic applications make the scheduling unit larger than one model invocation. Murakkab represents a workflow declaratively, separating the dependency graph from a particular model and hardware assignment. A profile-guided optimizer selects among model, batching, and placement options, while an adaptive runtime responds to changing conditions under accuracy, latency, energy, and cost objectives. Across the paper’s workflows, Murakkab uses up to 2.8 times fewer GPU resources, 3.7 times less energy, and 4.3 times less cost than compared systems while maintaining the specified service objective[6].
The word “up to” is essential. These are distinct maxima under the evaluated workflows, not a universal discount that procurement can apply to every agent. Murakkab needs alternate execution choices, representative profiles, and a meaningful end-to-end SLO. Its architectural value is that an agent workflow stops being an opaque chain of requests. Once dependencies and quality choices are visible, the cloud can optimize the whole path instead of making locally reasonable decisions at each model endpoint.
Alibaba’s Weave shows why phase information also matters inside reinforcement-learning post-training. Rollout is commonly memory-bound, while training is compute-bound. A fully disaggregated design assigns separate clusters but introduces bubbles at the on-policy dependency. A fully co-located design shares accelerators but repeatedly pays transition and interference costs. Weave forms co-execution groups, plans across groups stochastically, rotates work inside a group, and retains warm state in host memory. Its production-scale testbed pairs 328 H20 devices with an equal number of H800s. It reports 1.84 times the cost efficiency of standard disaggregation and 1.38 times that of a co-located baseline while meeting all tested SLOs[7].
Murakkab and Weave make a shared claim about heterogeneity. A GPU is not a stable unit of demand. The same application produces nodes or phases with different accuracy sensitivity, memory footprints, compute intensity, and synchronization barriers. A scheduler that knows only requested GPU count sees an average shape that never exists. The stronger scheduler sees a sequence and preserves the warm state needed for the next transition.
Fleet utilization needs an accounting of losses
Alibaba’s ASI trace is useful because it separates allocation from effective use. The study covers six months, 155,410 GPUs, 81 departments, multiple vendors and generations, and four operating modes spanning development runs, model training, latency-sensitive serving, and batch inference. Its central finding is that high demand can coexist with stranded capacity. GPUs become unusable through node-level fragmentation, CPU mismatch, network-locality requirements, and user-reserved headroom. Fractional-GPU fragmentation is comparatively small because sharing itself is rare. After defragmentation, 20.2% fewer nodes hold otherwise unusable slack; SpotGPU separately raises the allocation ratio from 68% to 93%[8].
That 93% is still not useful model progress. ByteDance’s reliability paper tracks the next loss. Across 778,135 jobs in three months, the dataset includes 44,184 failures and 9,582 restarts. The production system reports 97% effective training time ratio on jobs using as many as 9,600 GPUs, with no unproductive interval longer than 50 minutes. Hot update is 11.04 times faster than requeue at 16,384-GPU scale, warm standby is 10.87 times faster, and every-step in-memory checkpointing blocks for only 0.01 to 0.04 seconds while maintaining more than 99% model-FLOPS utilization in the reported tests[9].
The design prioritizes recovery over perfect diagnosis. Fast detection may evict more devices than the eventual root cause requires, while deeper analysis continues in the background. This is rational when thousands of healthy GPUs are waiting at a barrier. It also depends on a mature fleet, workload-faithful validation, spare capacity, and an engineering organization that can operate the recovery paths. A small cluster should copy the separation between detection, restoration, and diagnosis, not the exact threshold or standby ratio.
The combined accounting is a funnel. Installed capacity is reduced by electrical and thermal deliverability. Allocatable capacity is reduced by topology and memory constraints. Assigned capacity is reduced by queueing and fragmentation. Productive progress is reduced by bubbles, stragglers, and recovery. Adding more GPUs widens the top of the funnel. Cross-layer runtime work widens its narrowest stage.
Five design rules emerge across the papers
First, the control boundary moves upward. The best device setting can lose at cluster scale, the best cache hit can arrive too late, and the most efficient workflow node can make the entire DAG more expensive. Optimization should stop at the narrowest business objective that can still be measured, usually completed work under an SLO and a facility constraint.
Second, slack has a type. Electrical headroom, cooling margin, fabric bandwidth, free HBM, host-memory residency, queue gaps, and recovery reserves cannot be added into one capacity number. They have different locations, lifetimes, and actuators. A scheduler needs a resource graph with time, not a row of averages.
Third, observability is valuable only when it is paired with an action. Panel telemetry without coordinated throttling only reports risk. Cache metadata without a loading-aware scheduler only reports a nominal hit. Failure classification without a fast restart path only explains lost time. The operational question is always: what safe decision can this signal change before the constraint changes again?
Fourth, warm state is infrastructure. Strata preserves reusable cache blocks, Weave keeps phase state in host memory, and ByteDance prepares a replacement rather than rebuilding every dependency after failure. The economically important resource is often not raw bandwidth but continuity. Destroying state can convert a brief reassignment into minutes of idle fleet time.
Fifth, reported maxima are local evidence. A 58.8-times checkpoint result, five-times serving result, 4.3-times cost result, and 11.04-times recovery result have different denominators and workloads. They should not be ranked or multiplied. The comparable artifact is the mechanism, the tested operating envelope, and the condition under which the gain disappears.
What an operator should build next
The immediate requirement is a cross-layer accounting system for useful work. For each major workload, record facility power and cooling headroom, topology-qualified capacity, bytes waiting by memory tier, queue and phase state, productive accelerator time, and recovery loss. Keep the denominator with every percentage. “93% allocated” and “97% effective training time” describe separate stages and should remain separate.
The second requirement is an actuator registry. Every telemetry stream should identify who can act, at what scope, within what latency, and with which rollback. A rack controller can change a power cap in seconds. A scheduler can move a pending job but may not move 3.2 PB of live memory. A serving runtime can reorder requests but cannot invent SSD bandwidth. This registry prevents a dashboard from promising capacity that no control plane can deliver.
Procurement should request APIs, not only components. Ask whether the storage service accepts grouped intent, whether cache state is visible to the scheduler, whether power caps can be coordinated by job, whether topology and failure domains are exposed, and whether recovery preserves warm state. These interfaces determine whether future software can reclaim stranded capacity after the hardware purchase.
Lastly, evaluate infrastructure by completed, SLO-valid work per constrained facility hour. This metric includes the resource the business cannot quickly add, such as megawatts, cooled rack positions, or a scarce accelerator island. It also penalizes a fast system that repeatedly misses latency, restarts jobs, or strands devices behind topology. Peak FLOPS still describe a component. They no longer describe the product.
Limits of this review
The selection is intentionally industry-centered and limited to primary papers publicly available by August 28, 2026. It omits academic systems without a disclosed industry deployment context, confidential production systems, and papers that provide claims without enough mechanism or evaluation detail. Several 2026 papers were available as conference pages or preprints before proceedings publication. Hardware, workloads, baselines, and service objectives differ, so their speedups are not normalized.
Industry evaluations also reflect each organization’s fleet and incentives. A hyperscaler can amortize specialized control planes across a scale that a smaller operator cannot. Conversely, a public paper may omit the operational complexity or failure cases that determine total cost. The nine papers are therefore evidence for an architectural direction, not a universal reference configuration.
Sources and copyright
This article is an independent editorial synthesis prepared for Silicon & Systems. It restates facts and results from the nine primary sources in our own words. No source text, tables, or figures are reproduced. All three figures and the card image were created for this article. Rights to the source papers remain with their respective authors and publishers. Primary records are available from Meta on arXiv, Prometheus at IEEE, DeepSeek-V3 at ACM, AITURBO at USENIX, Strata at USENIX, Murakkab at USENIX, Weave at USENIX, Alibaba’s fleet study at USENIX, and ByteDance’s reliability paper at ACM.