Reinforcement-learning post-training alternates between two mismatched phases. Rollout generates tokens and is usually limited by memory bandwidth. Training updates the model and consumes dense compute. Separating them onto inference-oriented and training-oriented GPUs appears efficient, but synchronous on-policy learning creates a strict dependency: training waits for fresh trajectories, then rollout waits for updated weights. One paid cluster is idle during much of the other’s active period.
Weave, from HKUST, Alibaba, and collaborators, uses a different unit of scheduling[1]. It places several RL jobs into a co-execution group and fills one job’s dependency bubble with another job’s active phase. Rollout remains on H20 GPUs, training remains on H800 GPUs, and the on-policy barrier remains intact within each job. A two-level scheduler chooses compatible groups and runs their phases in round-robin order. Host DRAM retains each job’s state so the switch is warm rather than a model reload.
The message for capacity planning is that cheaper phase-specific GPUs do not create savings unless a scheduler can reclaim dependency bubbles without cold-loading state or violating each job’s slowdown limit. This review therefore treats phase occupancy, host-memory residency, and SLO attainment as one cost calculation, and keeps the production replay separate from results based on synthetic arrivals or assigned objectives.
Cheaper rollout GPUs do not guarantee a cheaper cluster
The paper’s two clusters each use a 400 Gb/s InfiniBand fabric, while traffic between them crosses a 20 Gb/s Ethernet link. An H800 costs 2.85× as much per hour as an H20 in the evaluation model. Disaggregation places compute-heavy training on H800 and memory-heavy rollout on H20. Nevertheless, the two-week replay shows that isolated disaggregation costs $0.94k per hour, more than the $0.71k per hour of the co-located veRL baseline. Hardware specialization loses when idle time exceeds its price advantage.
Asynchrony can remove the barrier, but it changes the learning regime. Fully asynchronous systems allow rollout to use older parameters, introducing sample staleness. Weave targets synchronous PPO, GRPO, DAPO, and other regimes where the phase dependency remains part of the accuracy and convergence contract. The scheduler must reclaim idle capacity without letting a training step consume trajectories from the wrong model version.
The global placement problem is difficult because jobs have different model sizes, response lengths, batch sizes, and phase ratios. Rollout duration is stochastic and long-tailed. Each worker also holds hundreds of gigabytes of weights, optimizer state, data-pipeline state, and environment state. Generic GPU time slicing observes a job that changes phase; it does not see a constrained two-machine schedule whose state is too large to reload on every turn.
Co-execution groups make the search tractable
When a job arrives, Weave considers existing groups before creating a new one. Direct packing uses idle rollout and training slots already paid for. Rollout scaling adds lower-cost H20 capacity when a group has spare training capacity but not enough inference bandwidth. Isolated provisioning is the fallback. Every candidate must keep all members within their slowdown SLO and keep their state within each node’s host-memory limit.
Admission uses a conservative duration that assumes every generated response reaches the configured maximum token length. This reserves enough capacity for the worst rollout and lets the runtime reclaim any shorter completion as slack. Saturated groups are discarded before detailed search. Because each group usually contains fewer than ten jobs, the online decision is nearly linear in active groups. In simulation, placement takes 591 ms for 2,000 concurrent jobs; a brute-force optimum becomes intractable beyond small cases.
Within a group, a cyclic round-robin schedule runs one rollout and one training phase for every member in a meta-iteration. The paper proves this order minimizes dependency bubbles under the model. Long-tail migration then moves the last few rollout requests onto a subset of homogeneous H20 GPUs, freeing the other devices for the next job. Across the microbenchmarks, this migration adds 1.06× to 1.28× throughput without changing the measured reward trajectory for the 7B, 14B, or 32B Qwen2.5 configuration.
Warm state is the condition for fine-grained switching
Cold loading takes up to 135.8 seconds for the evaluated 70B rollout and up to 116 seconds for training. A cold switch can reduce end-to-end throughput by as much as 45%. Weave pins jobs to group nodes and retains model weights and execution context in host DRAM. The warm path takes 0.9 to 1.9 seconds for rollout and 4.1 to 5.9 seconds for training across 7B to 70B configurations. The largest reported reduction in switching latency is 71.5×.
Memory limits the group size. On an eight-GPU node, cached rollout state ranges from 275.7 GB for 7B to 490.3 GB for the evaluated 32B configuration. Training state ranges from 240 to 520.4 GB. A node with 1 to 2 TB of host memory can therefore retain only about two to five jobs. This is why Weave creates bounded locality domains rather than allowing any job to run anywhere.
Model synchronization uses the topology explicitly. Weave sends one sharded copy of updated parameters across the slow inter-cluster link through parallel point-to-point streams, then broadcasts locally over the faster fabric. For transfers from eight H800s to eight H20s, synchronization is 7.87× to 8.33× faster than veRL across 7B to 32B models. At 16-to-16 GPUs, the gain remains 2.62× to 2.75×.
The production replay separates utilization from cost
The main replay contains 200 RL jobs from one tenant over two weeks. Qwen-family models range from 3B to 32B, and maximum responses span 4,000 to 32,000 tokens. A job lasts 27.9 hours on average. Each one receives an allowed slowdown sampled uniformly between 1× and 2× its solo runtime. The physical testbed has two 328-device pools: H20 for rollout and H800 for training.
Weave serves the trace for $510 per hour, or $188.8k total. The same workload costs $347.9k with isolated disaggregation and $262.8k with co-located veRL, giving 1.84× and 1.38× cost efficiency respectively while all assigned SLOs are met. Of 200 arrivals, 60.5% fit directly into existing groups, 15% require additional rollout capacity, and 24.5% create a new group.
Relative to isolated disaggregation, dependency bubbles fall by 24.4% on rollout GPUs and 43.1% on training GPUs. Peak H800 use falls from 328 to 152, while peak H20 use falls from 328 to 216. These counts explain the cost result better than a single utilization percentage. The scheduler removes more training idle time because the production mix is rollout-heavy.
Conservative planning has a cost and a boundary
Worst-case token lengths protect SLOs but can reserve more capacity than typical rollouts need. Weave depends on several concurrent jobs with complementary phase ratios; a small cluster or a single dominant job has fewer bubbles to exchange. Host memory must be large enough to retain several states, and the 20 Gb/s cross-cluster link makes topology-aware synchronization unusually valuable. A deployment with a different price ratio or link hierarchy should rerun the cost model.
The main replay also assigns slowdown SLOs rather than taking them from production requests. Large-scale optimality experiments synthesize RL job profiles on a Microsoft Philly arrival trace. The drift experiment keeps static placement cost no higher than 1.11× the impractical baseline that globally regroups jobs every hour. However, SLO attainment falls to 95.6% in the mixed case. Recovered jobs are admitted again from their last checkpoint; Weave isolates the failure but does not eliminate checkpoint and lost-iteration cost.
Fully asynchronous RL is outside the design boundary because it has no structural rollout-training bubble to reclaim. Conversely, the approach is not limited to strictly on-policy algorithms if a bounded off-policy method still preserves a dependency window. The defining condition is an idle phase visible to the scheduler, not the name of the RL algorithm.
RL capacity should be priced by phase, state, and barrier
An operator evaluating disaggregated post-training should record four quantities for every job: rollout and training duration distributions, state retained per node, synchronization bytes across each network tier, and permitted slowdown relative to solo execution. Average GPU utilization cannot show whether a new job complements the active mix or competes for the same phase.
Weave’s central result is conditional but important. Specialized hardware lowers cost only after the scheduler accounts for the barriers that specialization introduces. The valuable resource is not an H20 or an H800 in isolation. It is a pair of phase slots, enough resident state for a warm switch, and a schedule that turns both into completed on-policy updates.
SLO protection must include learning semantics
Weave preserves synchronous on-policy ordering inside each job, but an operational SLO should state more than slowdown. It should include which actor version generated each trajectory, the maximum permitted age, whether partial rollouts can be discarded or reassigned, and which barrier defines a valid update. The scheduler can then optimize idle phases without silently changing the algorithm.
Long-tail migration deserves the same check. Moving the final rollout requests to a smaller set of homogeneous devices can release capacity, but the resulting trajectories must remain associated with the correct policy and sampling configuration. The paper reports unchanged reward trajectories in its evaluated settings. A new model, environment, or sampling stack should repeat that validation rather than assume scheduling is semantically neutral.
Quality monitoring should compare learning curves, output distributions, and final evaluation at matched samples or tokens, not matched wall time alone. A faster system can complete more updates and appear better even if each update is slightly worse. Separating systems throughput from sample efficiency keeps the performance claim honest.
Resident state is a capacity reservation
Warm switching depends on hundreds of gigabytes of host state per job. That memory is not incidental cache. It is reserved capacity that enables the scheduling quantum. Operators should account for weights, optimizer state, execution context, pinned buffers, and copies required for synchronization, then include memory bandwidth and NUMA locality as well as byte capacity.
Contention on host memory can lengthen several switches at once and invalidate the schedule. A group may fit by bytes but fail by bandwidth when jobs transition together. Admission should model peak simultaneous movement and preserve headroom for recovery. Telemetry should show warm-state hit rate, switch latency distribution, bytes evicted, and jobs forced into cold loading.
Failure can remove both a compute device and the warm state that made rapid switching possible. Replication or checkpoint policy should cross the same failure domains used by the GPU pools. The recovery SLO must include restoring resident state or operating temporarily with a slower schedule. Otherwise, the cluster is efficient only while every host holding the working set remains healthy.
The price model must travel with the result
Weave’s dollar outcome depends on the H800-to-H20 price ratio, pool sizes, 20 Gb/s cross-cluster link, workload phase mix, and assigned slowdown range. A deployment should rebuild the model with local prices and topology rather than apply the reported 1.84×. It should also include host DRAM, network, checkpoint capacity, and stranded reserve.
The comparison needs identical service obligations. Isolated disaggregation, colocation, and co-scheduling should receive the same jobs, arrival times, convergence rules, slowdown limits, and failure treatment. Peak device count and total accelerator-hours should appear beside dollar cost. These labels reveal whether savings come from cheaper hardware, fewer devices, better utilization, or a relaxed obligation.
Sensitivity analysis is more valuable than one optimum. Change the rollout-to-training ratio, job concurrency, response-length tail, price ratio, link rate, and host-memory limit. The region in which co-scheduling remains beneficial tells an operator which future drift it can tolerate and which threshold should trigger a new grouping plan.
Rollout should begin with reversible capacity
The first production use can admit preemptible or lower-priority post-training jobs whose checkpoints and deadlines tolerate rescheduling. The scheduler runs in recommendation mode, comparing its group plan with the existing placement. Once timing predictions and warm-switch behavior are calibrated, it can control a bounded pair of pools before expanding.
Operators should inject a failed rollout node, failed training node, cross-cluster link slowdown, and loss of host state. The system must preserve policy-version correctness, recover from a trusted checkpoint, and prevent one group from consuming all spare capacity. These tests expose whether the elegant phase schedule survives ordinary fleet events.
The control policy should also publish why a job was placed directly, expanded with rollout devices, or assigned a new group. An explanation tied to predicted phase time, memory, SLO, and cost makes unexpected behavior diagnosable and supports later audit of the price model.
We read Weave as a method for turning dependency bubbles into a schedulable resource without weakening on-policy learning. Its transferable result is not a universal hardware pairing. It is the joint accounting of phase time, resident state, synchronization, price, and barrier semantics. A deployment succeeds when that accounting lowers the cost of completed valid updates under its own workload and failure distribution.
Source and attribution
This article is an editorial summary prepared by Silicon & Systems. It restates the cited paper’s mechanisms, evaluation conditions, results, and limitations in our own words. No sentences, tables, or figures from the paper are reproduced; all three figures and the card image were created for this article from reported facts. Copyright (c) 2026 the authors. The paper is publicly available through the USENIX OSDI 2026 page.