Neural Scaling Laws: The Proof Comes From Rydberg Atoms

An ablation study that takes apart the data, not the model

An ablation study on a language model usually takes apart the architecture: attention heads, layers, tokenizer. The work filed on arXiv on 17 September 2026^[1] by David S. Berman, Ying-Jer Kao, Roger G. Melko and Alexander G. Stapleton takes apart the data instead, and the result concerns anyone who trains language models.

The authors study the neural scaling laws of RydbergGPT, an autoregressive transformer trained on projective qubit measurement data collected from arrays of interacting Rydberg atoms. The quantum system shows a finite-size remnant of a critical point as the laser detuning is varied. Ten pages, six figures, one tight thesis.

The experimental lever sits outside the model. The transformer stays fixed, and what changes is the source feeding it.

Two loss regimes around the critical point

The evidence shows two distinct behaviours. Near the critical point, loss as a function of training dataset size is well described by a power law with a loss floor correction. Away from criticality, the quality of that description drops substantially.

The point deserves some patience. The power law with a loss floor is the form the industry uses to plan compute: it says how much the loss improves every time the dataset doubles, and where the gains stop.

When that form loses its grip, planning loses its instrument. The curve can still be drawn, and it stops being a reliable guide for deciding how much to spend.

The two-point function between qubits and language

The second half of the work compares the statistical structure of Rydberg measurements with that of natural language corpora. The instrument is a "two-point" function built on mutual information, normalised by entropy and corrected for finite sample size.

The finite-sample correction is the methodological detail worth noting. Mutual information estimated on limited data carries an upward bias, and anyone who measures it while ignoring that bias mistakes noise for structure.

The comparison returns a clear ordering. Statistics near criticality are the closest to those of natural language. Qubit configurations far from the critical point show two-point functions that decay more rapidly.

Why correlation decay matters

Rapid decay says something precise: the dependence between two positions vanishes after just a few steps. The useful signal lives entirely at short range, and a large model soon exhausts what it can learn.

Language behaves differently. The relationship between words survives across long scales, within the sentence, within the paragraph, within the entire document.

The authors read the two results together and frame the point as a supported hypothesis: multi-scale dependence contributes to a stable scaling law, and scaling behaviour should be seen as a property of the model-data pair. The architecture remains one half of the equation. The other half is the corpus.

What the public abstract leaves out

It is worth stating plainly what this reading lacks. The abstract reports the relationship in qualitative form and stays silent on the values that make a scaling claim verifiable.

• number of measurement samples for each detuning point
• how many values of the detuning parameter were explored
• range of dataset sizes covered by the curves
• estimated power law exponent and loss floor value
• goodness-of-fit measure used to judge the loss of quality away from criticality

These numbers live in the six figures of the full text. A judgement on the strength of the result requires that reading, and the direction of the finding remains legible from the public material regardless.

Anyone assessing the reach of this work should make the same move the authors themselves made: a controlled physical system, a fixed model, one variable moved by hand.

The R&D budget and the implicit premise of scaling

An investment committee approving spend on training compute often starts from an implicit premise: more data and more parameters produce a predictable improvement. This evidence shifts that premise by one step.

The predictability observed here depends on the statistical structure of the source. Where that structure carries long-range correlations, the loss curve stays legible and spending becomes plannable. Where the structure is poor, the curve loses shape and the budget buys an uncertain gain.

The diagnosis is simple to state. The question "how well does our model scale" remains badly posed, and the measurable version sounds like this: how well does our model scale on this corpus.

The data infrastructure that makes the question measurable

The two-point function used by the authors can be computed on any tokenized corpus. It requires access to the raw data and a pipeline that preserves sequence order, along with an entropy estimate corrected for finite sample size.

The cost of that computation is modest compared with a training run. The signal, in principle, arrives before the spend.

For a head of analytics the practical point is this: the dependence structure of internal data becomes a measurable indicator, on a par with its volume. An archive of tickets, contracts or logs has a correlation profile of its own, and that profile changes from domain to domain. Measuring it before fine tuning costs a few hours of compute.

The board thesis that needs revising

Many board-level technology theses rest on a reading of the architecture: the transformer scales, so the race is for the model. This evidence supports a narrower reading, in which what scales is the pairing of transformer and data with multi-scale dependence.

The difference has consequences for allocation. The first reading treats data as a commodity and the architecture as the scarce good; the second reverses the order and makes the proprietary corpus the asset whose structure must be measured.

One boundary remains to be respected, set by the authors themselves: a specific physical system, a specific model, a supported hypothesis. Extending the result to a rule about enterprise corpora goes beyond that boundary. The defensible reading today is this: there exists a measure of data structure that correlates with the stability of the scaling law in a controlled case, and the rest has to be measured.