AI interpretability research: measuring the noise floor nobody reports

A cosine of 0.996 that stays unreadable

A paper filed to arXiv on 12 August 2026 measures the quantity that AI interpretability research omits when it certifies that an artifact holds up under quantization. That quantity is the estimator's noise floor.

The practice under scrutiny is widespread. An interpretable direction is calibrated on full-precision weights, then transferred onto quantized weights. The match between the two versions is certified with scale-invariant statistics: cosine similarity, correlation, AUROC.

The work is authored by Pranav Varshney. A cosine of 0.996 between a full-precision refusal direction and its quantized counterpart stays unreadable as long as the value of n at which it was computed is missing (arXiv:2609.30275^[1]). That value turns out to be absent from the original publication.

The main text runs to five pages, with one figure and three tables.

κ = nρ²/d: the missing quantity

For the difference-in-means estimator, the split-half floor depends on a single dimensionless number: κ = nρ²/d. There are three ingredients.

• n: the size of the sample used to estimate the direction
• ρ: the class separation measured on the activations
• d: the dimensionality of the activation space

The closed form E[cos] ≈ (1+4/κ)⁻¹ belongs to classical statistics. The ingredient that is missing is ρ, which requires a measurement on the actual activations.

The author states he knows of zero transfer-under-compression studies that report that value.

The result is a κ the reader has no means of computing. With κ unknown, even the expected agreement between two independent runs of the same estimator remains unknown. A high cosine therefore becomes compatible both with preservation of the direction and with pure sampling chance.

The published number still looks like evidence, and its evidentiary function has already evaporated.

ρ from 33 to 61 across network depth

On the Qwen2.5-1.5B-Instruct model, the class separation measured across depth ranges from 33 to 61.

At those values, two independent runs of the same estimator agree between 0.978 and 0.994 through sampling alone, everything else held equal. The weights stay identical: what is in play is the intrinsic variability of the estimate.

The gap between 0.994 and 0.996 is where the entire evidentiary claim of current practice lives. Two thousandths.

The direction of the measurement matters. High class separation makes the estimator very stable, and a stable estimator produces cosines close to 1 even when comparing two random samples of the same phenomenon. The stability of the instrument gets read as stability of the object being measured.

The paper ships with code, data and a single-cell reproduction; its open discussion page sits on alphaXiv^[2].

The null has to be measured inside the quantized model

Where n is known, the paper judges every low-precision cosine against the split-half null measured inside that quantized model.

The rationale is methodological. A null computed at full precision assumes the low-precision estimator has the same variance, and that assumption is exactly what a null exists to test.

Anyone who adopts the wrong null gets a test that confirms the starting hypothesis. Quantization reduces the numerical precision of the activations, and reduced precision raises the noise of the estimate. A floor calibrated on the wrong model ends up too low, and every observed deficit looks significant.

This desk finds here the most interesting anomaly in the work: the correction requires a measurement inside the already-compressed artifact, a step the literature skips out of habit. The choice of reference decides the outcome before the data do.

INT4 rotates, INT8 stays indistinct

With the correct null, the two quantization regimes separate. At INT4 the direction rotates, and the deficit exceeds the estimator's own noise.

At INT8 the work detects zero movement. The author notes that the absence of detected movement differs from a claim of equivalence.

The distinction carries weight. A test with insufficient power produces the same outcome as a test that observes preservation, and the two outcomes have opposite implications for anyone deciding on a deployment. Conflating them means reading the limits of the instrument as a property of the model.

Context makes the result concrete. LLM.int8() (arXiv:2208.07339^[3], 2022) brought 8-bit matrix multiplication into production for large-scale transformers, and the move to 4 bits is today the economical choice for serving models on constrained hardware.

Translation and attenuation call for opposite remedies

A scale-invariant statistic ignores scale by construction.

The paper shows that a measure of this kind conflates translation with attenuation of a transferred decision variable. The two pathologies call for opposite interventions: the first is corrected on the intercept, the second on the gain.

The practical implication concerns refusal systems. A translated direction goes on separating cases with the same quality, and recalibrating the threshold is enough. An attenuated direction has lost margin, and the recalibrated threshold shifts the problem from false negatives to false positives.

An identical cosine covers both scenarios. Choosing the remedy therefore becomes a bet dressed up as a diagnosis.

The work closes with reporting recommendations whose stated cost is a single forward pass: report n, report ρ, report the floor measured in the target model.

A century between the theorem and its application

The correction for attenuation has a precise paternity. Charles Spearman's classic on the measurement of association between two things, from 1904 and republished in the International Journal of Epidemiology (doi:10.1093/ije/dyq191^[4]), shows that the reliability of an instrument places a ceiling on the observable correlation.

From that ceiling follows the split-half logic: you measure the instrument's agreement with a replica of itself, and that value becomes the reference for every subsequent correlation. Psychometrics has applied the rule for generations.

The interpretability field adopted the statistic and left the correction behind. The plausible reason is one of incentives: a high cosine published without a floor supports a safety claim, whereas the same cosine with a floor cuts it down to size. The evidence shows that the floor, where it is measured, reaches 0.994.

A number the field's tables have left out for years.

What changes for whoever allocates the R&D budget

For an investment committee the consequence concerns the quality of the evidence, never the pace of the spending. A safety pipeline that certifies its own artifacts with floorless statistics generates documentation, and documentation then gets read as verification.

For whoever runs analytics the infrastructure requirement is modest and specific: retain the activations and the count of samples used to estimate each direction, inside the model that goes into production.

For a board the thesis to revisit is the one that treats quantization as a step neutral with respect to model behaviour. At 4 bits the evidence shows a measured rotation. At 8 bits it shows the silence of current instruments, which is a different thing.

The perimeter of the result stays the one the author declares: a 1.5-billion-parameter model, one estimator, two bit regimes. Extending the conclusion to other architectures requires the same measurement, repeated.