“Write once, run anywhere” is difficult for tensor programs because the source code is partly an architecture description. A CUDA kernel exposes SIMT indices, shared-memory staging, synchronization, and NVIDIA-specific instructions. A Cambricon BANG C kernel maps work through a different SIMD model. CPU VNNI and AMD HIP introduce other vector widths, memory behaviors, and intrinsic contracts. Translating syntax without reconstructing these decisions can produce code that compiles yet computes the wrong answer.
Rule-based translators are dependable inside the cases their authors encoded, but each source-target pair needs expert rules. Symbolic synthesis can search for equivalent code, although the search space grows too quickly for a complete accelerator program. A large language model covers more syntax and APIs, but its output is probabilistic. In the paper’s motivating CUDA-to-BANG experiment, zero-shot GPT-4 produced compilation errors in every case. Few-shot prompting compiled more often, yet 92.3% of outputs had computation errors[1].
QiMeng-Xpiler combines these incomplete tools. The LLM proposes large structural changes; unit tests identify failures; symbolic synthesis repairs a small localized region; and an outer search chooses transformations and tuning parameters. The design treats neural generation as a search-space reducer rather than a correctness authority.
Eleven passes expose the architecture changes
The transcompiler first decomposes migration into 11 transformations. Loop recovery turns parallel built-ins into sequential loops. Loop binding maps loops onto the target’s parallel variables. Split, fuse, reorder, expansion, and contraction reshape iteration. Cache and pipeline passes adapt memory movement. Tensorization replaces loop bodies with special instructions, while detensorization expands a source intrinsic back into ordinary operations.
This vocabulary separates three semantic gaps: parallel execution, memory hierarchy, and tensor instructions. A single prompt must solve all three at once and can easily change an index while moving a buffer. A pass limits the kind of change under consideration and provides a narrower unit test boundary. It also makes target-manual retrieval useful because the prompt can request only the APIs and constraints relevant to the current step.
For each pass, the system asks an LLM to annotate the source with computation and hardware information. It searches programming manuals with BM25 and feeds selected material into predefined meta-prompts. The generated candidate then runs against tests. Successful code moves to the next pass; failing code enters repair.
Pass decomposition has another benefit: it creates a graph of intermediate programs. One sequence may detensorize, reorganize loops, and tensorize for the destination. Another may preserve more of the original shape. The compiler can measure candidates instead of committing to the first semantically plausible route.
Symbolic repair is deliberately local
An SMT solver cannot tractably synthesize a complete 200-line tensor operator from unconstrained behavior. QiMeng-Xpiler uses the LLM-generated candidate and its error as a localization hint. It extracts the involved buffers and expressions, constructs a code sketch, and asks the solver to fill a small hole that satisfies the examples. The repaired snippet is stitched back into the candidate and tested again.
This division of labor is the paper’s central claim. The LLM supplies broad, target-specific structure without proving it. The solver handles a bounded algebraic or indexing relation where exhaustive reasoning is feasible. Tests connect the two. Ablation results support the combination: removing SMT lowered computation accuracy sharply in difficult directions, and adding LLM self-debugging did not recover the gap. For HIP-to-BANG C, the no-SMT variants stayed near 52.4%, while the full system reached 86.9%.
The word “correctness” still needs care. The implementation calls an output correct when it passes its unit-test set on hardware. SMT repair constrains a localized sketch against examples; it does not prove whole-program equivalence for every shape, aliasing pattern, floating-point corner, or race. A production compiler must distinguish test-suite success from a formal guarantee and retain validation for every deployed shape.
Undefined behavior is another boundary. If a source kernel relies on an undocumented warp property or a race that happens to be stable, semantic preservation is not well-defined. Manual and compiler versions also change intrinsic behavior. The generated artifact should therefore be tied to source revision, target toolchain, hardware, retrieved documentation, prompts, tests, and search seed.
Performance requires searching transformations, not only code
Functionally correct tensor code can be unusably slow. QiMeng-Xpiler applies hierarchical auto-tuning. Intra-pass search enumerates parameters such as tile sizes. Inter-pass search uses Monte Carlo tree search to choose which transformation follows which. Executing a candidate supplies a latency reward that propagates to its ancestors, biasing later exploration toward useful sequences.
This makes compilation an empirical optimization job. It can discover that a target needs a different order of caching, loop restructuring, and tensorization than the source. It also means results depend on the benchmark shapes and device used during search. A program tuned on eight representative shapes may not be optimal for a new batch, sequence length, sparsity pattern, or thermal state.
The cost is substantial. CUDA-to-BANG C compilation for six studied operators took 1.2 to 7.8 hours, averaging 3.7 hours. Complex matrix multiplication spent more time in auto-tuning; operators with many special intrinsics increased both generation and repair work. This is suitable for an offline kernel migration pipeline, not just-in-time compilation.
Search budgets create a product decision. More trials can improve performance but consume accelerator time and LLM service capacity. Organizations need a stopping rule based on expected invocation volume. A kernel used millions of times may justify hours of search, while a rarely used operator may be cheaper to run through a portable framework fallback.
Four programming models expose the result
The evaluation spans four targets: Intel Gold 6348/VNNI, NVIDIA A100/CUDA, AMD MI200/HIP, and a Cambricon MLU programmed in BANG C. It uses 21 operators in six groups, including matrix multiplication, convolution, activation, pooling, elementwise, and LLM operations. Eight shapes drawn from models such as GPT, LLaMA-2, BERT, ResNet, and MobileNet produced 168 cases. Source programs ranged from 7 to 214 lines.
Across the reported translation directions, compile success was close to 100%, while computation accuracy ranged from 86.9% to 100%. CUDA C to HIP reached 100% in both measures, compared with 85.7% for HIPIFY in the selected suite. C-to-CUDA computation accuracy was 98.2%, roughly 50 percentage points above the PPCG baseline. The more difficult CUDA-to-BANG direction reached 91.7% computation accuracy.
The remaining failures matter. A 95% average is an impressive research result but means one in twenty tested cases did not pass the functional criterion. A transcompiler cannot silently ship the other five percent. Its workflow needs an explicit unresolved state, minimized failing tests, and a manual repair path. The output is an engineering candidate, not automatically a releasable kernel.
Performance averaged 0.78 times the manually optimized PyTorch backends across four directions. Handwritten libraries retained an advantage from assembly, deeper software pipelines, aggressive unrolling, and carefully constructed shared-memory movement. FlashAttention variants translated across platforms achieved 0.61 to 0.81 times native implementations. The compiler lowers porting cost; it does not erase the value of vendor kernel teams.
A Deformable Attention case quantified productivity. Transcompilation improved measured development productivity by 34.3 times for GPU and 96.0 times for MLU. The MLU output still needed debugging: junior and senior programmers spent another three hours and half an hour, respectively. That is a useful operational picture because it counts human completion rather than treating generation as finished work.
A compiler pipeline needs release gates
The first gate is functional coverage. Tests should include destination-specific limits, odd shapes, boundary tiles, alignment, accumulation precision, and concurrent execution. Metamorphic relations can add coverage when exact outputs are expensive. Differential testing against the source backend should use both tolerance and adversarial values.
The second gate is performance coverage. Average normalized speed can hide one operator or shape that regresses badly. Teams should record latency distributions, memory consumption, compilation resources, and fallbacks per shape. If an output does not meet the threshold, the framework can retain a vendor library or source-platform service rather than accepting the generated kernel.
The third gate is reproducibility and audit. Every transformation and repair should be replayable, with intermediate code and test outcomes stored. A new LLM, manual revision, compiler, or device should trigger the relevant subset of validation. Otherwise the apparent productivity gain becomes an opaque binary whose provenance cannot be reconstructed.
QiMeng-Xpiler’s reusable insight is not that LLM output has become trustworthy by itself. It is that a probabilistic generator can make symbolic work small enough to apply, while structured passes and empirical search turn one large translation into auditable decisions. The combination expands the platform range, but the reported accuracy and speed show why human review, strong tests, and optimized-library fallbacks remain part of the system.
Source and copyright notice
This article is an editorial analysis by Silicon & Systems. It restates the method, measurements, and limitations in our own words. No source sentence, table, or figure is reproduced; the figure was created for this article. The paper is available from the USENIX OSDI 2025 presentation page. Copyright remains with the authors, 2025.