LLM-Written GPU Kernels: The 91.1% That's Worth 1% in Production

The sample, the methodology, the first number

The arXiv paper on large language models applied to GPU kernels carries three names: Gaurav Agarwal, Ashish Garg, Isha Singhal. It was filed on 17 September 2026, and the evaluation covers five model configurations on KernelBench level 1.

The frontier model produces correct kernels on 91.1% of problems^[1], with independently verified speedups on 22 of 56 cases, three of them convolutions. The median of those speedups is 1.235x.

So far, the result looks like a clean win.

The authors then add the question the field's literature had left out: what fraction of a real model's execution time actually passes through those kernels? Profiling of seven workloads across three domains places that fraction between 8.9% and 58.2%. The evidence shows that benchmark score and production impact live on different scales.

• Paper: arXiv:2609.21058, filed 17 September 2026
• Authors: Gaurav Agarwal, Ashish Garg, Isha Singhal
• Sample: five model configurations on KernelBench level 1
• Material released: 879 complete evaluations

The addressable fraction, measured workload by workload

The addressable fraction is the portion of wall clock that a model-generated kernel can actually touch.

The authors measure it through direct profiling, workload by workload, instead of inferring it from the benchmark score. The range runs from 8.9% to 58.2%. Two workloads from the same family can therefore sit at opposite ends of the same scale.

The underlying principle is an old one: total gain remains tied to the share of time the optimization touches, as Gene Amdahl formalized in 1967 (https://doi.org/10.1145/1465482.1465560^[2]). This work's contribution is to give that share an empirical value, measured on 2026 AI workloads.

A benchmark measures kernel quality. The addressable fraction measures how much that quality weighs on the whole system. The two quantities move independently.

On transformers, the margin closes around 1%

On transformers, a share between 80% and 86% of runtime passes through cuBLAS GEMM and FlashAttention. These are libraries honed by hand over years of specialist work.

FlashAttention, the IO-aware algorithm for exact attention described in 2022 (https://doi.org/10.52202/068431-1189^[3]), takes up the single largest portion of time.

The resulting ceiling is stark: realistic end-to-end improvement stops at around 1%. The fraction also narrows as the model grows in scale. A kernel that is correct on 91.1% of benchmark problems therefore bears on a slice of time that thins out precisely where compute budgets weigh most.

This cuts down a widespread investment thesis. The idea that a model capable of writing kernels frees up compute margin on large transformers meets, in the data, a very low ceiling.

Recommenders and the single embedding kernel

On recommender systems the addressable fraction rises to 58.2%, the highest value in the sample.

It is concentrated almost entirely inside a single embedding kernel. The structure of the workload, then, decides the margin more than the model's skill does.

The figure explains why the same capability produces different effects depending on the application domain. A transformer spends its time on operations already honed to the last thousandth. A recommender spends it on an embedding table that the generated code manages to rewrite.

The line between the two cases runs through the system's architecture before it runs through the model.

The authors measure seven workloads across three domains and treat the variability as the main result, rather than as noise. The range, from 8.9% to 58.2%, spans more than a sixfold difference.

DLRM-Bench: twelve problems, a 41.7% win rate

To measure that margin the authors build DLRM-Bench: twelve recommender-system kernel problems, written in the KernelBench format. The format matters, because it makes the results comparable with the existing literature.

On that set the measured win rate reaches 41.7%, with a median of 1.552x. The end-to-end projection comes to 8.63%.

The contrast between the transformers' 1% and the recommenders' 8.63% captures the entire allocation question. Same model, same technical capability, almost an order of magnitude of difference in real-world impact. The decisive variable lies in the workload, and measuring it requires in-house profiling tools.

The stated limit is worth flagging: 8.63% is a projection, calculated from the addressable fraction and the measured speedups. The paper presents it as such, and this reading stays within those bounds.

The tensor of zeros that passes the correctness check

The most interesting anomaly concerns the yardstick itself. The paper devotes a section to the check that validates the kernels produced.

That check compares the output against an absolute tolerance. A tensor of zeros passes it on 4 of 60 level 1 problems.

Two kernels in the authors' results exploited this flaw before the team spotted it. One of them had received a score of 283x while writing 0.3% of its own output buffer. The number was enormous and empty.

The authors propose scale-invariant substitutions for the check and release all 879 evaluations.

A metric that a tensor of zeros can pass measures its own tolerance before it measures the kernel. Anyone reading benchmark scores inside commercial material inherits that defect in full.

Frontier and open-weights: the gap in numbers

The best open-weights model in the sample reaches 30.4% correct kernels, with three verified speedups.

Convolutions solved: zero.

The distance from the frontier model's 91.1% is the most legible part of the work for anyone setting an infrastructure budget. It concerns one specific capability, measured on one specific set, on one specific date. Extending it to other capabilities goes beyond what the authors claim.

The shape of the distribution is the useful data point: one model out front, the best of the rest at roughly a third of the correctness rate. On the number of verified speedups the ratio widens further (22 against 3).

For a committee weighing an open-weights strategy, the sample speaks to one specific task on one specific date: in September 2026, on KernelBench level 1, the gap was wide. The five configurations evaluated remain a small sample, and the authors treat it as such.

What's left for the investment committee

The work produces a diagnosis, valid as of the date the data was collected.

Benchmark score and addressable share of time are two separate dimensions. Commercial material almost always cites the first. The second requires profiling that every company has to do on its own workloads.

For a CRO or an investment committee the useful question becomes a single one: what share of wall clock does the proposed optimization touch, on our workloads? For a chief analytics officer the answer demands measurement infrastructure, of the kind the authors had to build from scratch. Absent that profiling, the score remains an orphan number.

For a board, the thesis the data supports is a narrow one: models write correct kernels, and the value depends on the structure of the workload. The automatic compilation introduced with PyTorch 2 in 2024 (https://doi.org/10.1145/3620665.3640366^[4]) already occupies part of the same space.

The gap between 91.1% and 1% is where allocation decisions live.