Reinforcement-learning post-training alternates between two mismatched phases. Rollout generates tokens and is usually limited by memory bandwidth. Training updates the model and consumes dense compute. Separating them onto inference-oriented and training-oriented GPUs appears efficient, but synchronous on-policy learning creates a strict dependency: training waits for fresh trajectories, then rollout waits for updated weights. One paid cluster is idle during much of the other’s active period.
Weave, from HKUST, Alibaba, and collaborators, uses a different unit of scheduling[1]. It places several RL jobs into a co-execution group and fills one job’s dependency bubble with another job’s active phase. Rollout remains on H20 GPUs, training remains on H800 GPUs, and the on-policy barrier remains intact within each job. A two-level scheduler chooses compatible groups and runs their phases in round-robin order. Host DRAM retains each job’s state so the switch is warm rather than a model reload.
Cheaper rollout GPUs do not guarantee a cheaper cluster
The paper’s two clusters each use a 400 Gb/s InfiniBand fabric, while traffic between them crosses a 20 Gb/s Ethernet link. An H800 costs 2.85× as much per hour as an H20 in the evaluation model. Disaggregation places compute-heavy training on H800 and memory-heavy rollout on H20. Nevertheless, the two-week replay shows that isolated disaggregation costs $0.94k per hour, more than the $0.71k per hour of the co-located veRL baseline. Hardware specialization loses when idle time exceeds its price advantage.
Asynchrony can remove the barrier, but it changes the learning regime. Fully asynchronous systems allow rollout to use older parameters, introducing sample staleness. Weave targets synchronous PPO, GRPO, DAPO, and other regimes where the phase dependency remains part of the accuracy and convergence contract. The scheduler must reclaim idle capacity without letting a training step consume trajectories from the wrong model version.
The global placement problem is difficult because jobs have different model sizes, response lengths, batch sizes, and phase ratios. Rollout duration is stochastic and long-tailed. Each worker also holds hundreds of gigabytes of weights, optimizer state, data-pipeline state, and environment state. Generic GPU time slicing observes a job that changes phase; it does not see a constrained two-machine schedule whose state is too large to reload on every turn.
Co-execution groups make the search tractable
When a job arrives, Weave considers existing groups before creating a new one. Direct packing uses idle rollout and training slots already paid for. Rollout scaling adds lower-cost H20 capacity when a group has spare training capacity but not enough inference bandwidth. Isolated provisioning is the fallback. Every candidate must keep all members within their slowdown SLO and keep their state within each node’s host-memory limit.
Admission uses a conservative duration that assumes every generated response reaches the configured maximum token length. This reserves enough capacity for the worst rollout and lets the runtime reclaim any shorter completion as slack. Saturated groups are discarded before detailed search. Because each group usually contains fewer than ten jobs, the online decision is nearly linear in active groups. In simulation, placement takes 591 ms for 2,000 concurrent jobs; a brute-force optimum becomes intractable beyond small cases.
Within a group, a cyclic round-robin schedule runs one rollout and one training phase for every member in a meta-iteration. The paper proves this order minimizes dependency bubbles under the model. Long-tail migration then moves the last few rollout requests onto a subset of homogeneous H20 GPUs, freeing the other devices for the next job. Across the microbenchmarks, this migration adds 1.06× to 1.28× throughput without changing the measured reward trajectory for the 7B, 14B, or 32B Qwen2.5 configuration.
Warm state is the condition for fine-grained switching
Cold loading takes up to 135.8 seconds for the evaluated 70B rollout and up to 116 seconds for training. A cold switch can reduce end-to-end throughput by as much as 45%. Weave pins jobs to group nodes and retains model weights and execution context in host DRAM. The warm path takes 0.9 to 1.9 seconds for rollout and 4.1 to 5.9 seconds for training across 7B to 70B configurations. The largest reported reduction in switching latency is 71.5×.
Memory limits the group size. On an eight-GPU node, cached rollout state ranges from 275.7 GB for 7B to 490.3 GB for the evaluated 32B configuration. Training state ranges from 240 to 520.4 GB. A node with 1 to 2 TB of host memory can therefore retain only about two to five jobs. This is why Weave creates bounded locality domains rather than allowing any job to run anywhere.
Model synchronization uses the topology explicitly. Weave sends one sharded copy of updated parameters across the slow inter-cluster link through parallel point-to-point streams, then broadcasts locally over the faster fabric. For transfers from eight H800s to eight H20s, synchronization is 7.87× to 8.33× faster than veRL across 7B to 32B models. At 16-to-16 GPUs, the gain remains 2.62× to 2.75×.
The production replay separates utilization from cost
The main replay contains 200 RL jobs from one tenant over two weeks. Qwen-family models range from 3B to 32B, and maximum responses span 4,000 to 32,000 tokens. A job lasts 27.9 hours on average. Each one receives an allowed slowdown sampled uniformly between 1× and 2× its solo runtime. The physical testbed has two 328-device pools: H20 for rollout and H800 for training.
Weave serves the trace for $510 per hour, or $188.8k total. The same workload costs $347.9k with isolated disaggregation and $262.8k with co-located veRL, giving 1.84× and 1.38× cost efficiency respectively while all assigned SLOs are met. Of 200 arrivals, 60.5% fit directly into existing groups, 15% require additional rollout capacity, and 24.5% create a new group.
Relative to isolated disaggregation, dependency bubbles fall by 24.4% on rollout GPUs and 43.1% on training GPUs. Peak H800 use falls from 328 to 152, while peak H20 use falls from 328 to 216. These counts explain the cost result better than a single utilization percentage. The scheduler removes more training idle time because the production mix is rollout-heavy.
Conservative planning has a cost and a boundary
Worst-case token lengths protect SLOs but can reserve more capacity than typical rollouts need. Weave depends on several concurrent jobs with complementary phase ratios; a small cluster or a single dominant job has fewer bubbles to exchange. Host memory must be large enough to retain several states, and the 20 Gb/s cross-cluster link makes topology-aware synchronization unusually valuable. A deployment with a different price ratio or link hierarchy should rerun the cost model.
The main replay also assigns slowdown SLOs rather than taking them from production requests. Large-scale optimality experiments synthesize RL job profiles on a Microsoft Philly arrival trace. The drift experiment keeps static placement cost no higher than 1.11× the impractical baseline that globally regroups jobs every hour. However, SLO attainment falls to 95.6% in the mixed case. Recovered jobs are admitted again from their last checkpoint; Weave isolates the failure but does not eliminate checkpoint and lost-iteration cost.
Fully asynchronous RL is outside the design boundary because it has no structural rollout-training bubble to reclaim. Conversely, the approach is not limited to strictly on-policy algorithms if a bounded off-policy method still preserves a dependency window. The defining condition is an idle phase visible to the scheduler, not the name of the RL algorithm.
RL capacity should be priced by phase, state, and barrier
An operator evaluating disaggregated post-training should record four quantities for every job: rollout and training duration distributions, state retained per node, synchronization bytes across each network tier, and permitted slowdown relative to solo execution. Average GPU utilization cannot show whether a new job complements the active mix or competes for the same phase.
Weave’s central result is conditional but important. Specialized hardware lowers cost only after the scheduler accounts for the barriers that specialization introduces. The valuable resource is not an H20 or an H800 in isolation. It is a pair of phase slots, enough resident state for a warm switch, and a schedule that turns both into completed on-policy updates.
Source and attribution
This article is an editorial summary prepared by Silicon & Systems. It restates the cited paper’s mechanisms, evaluation conditions, results, and limitations in our own words. No sentences, tables, or figures from the paper are reproduced; all three figures and the card image were created for this article from reported facts. Copyright (c) 2026 the authors. The paper is publicly available through the USENIX OSDI 2026 page.