On-Device Inference: AI Leaves the Cloud for the Phone

The thesis: value moves inside the device

Frontier compute has stopped buying competitive advantage.

On 15 September 2026 a team of researchers from LISN and STL published a measurement that carries more weight than most keynotes. A 0.8-billion-parameter model reaches 89.7% accuracy on the SNLI dataset, 1.9 points off its twin trained on text, as documented in the paper by Boufouss and colleagues^[1].

The figure deserves attention for a specific reason. The pipeline excludes raw text entirely: the classifier reads graphs of atomic propositions, translated into ConceptNet triples. The two modalities together climb to 92.1% on the same dataset.

This is a regime change. Capability per parameter has crossed the threshold beyond which gigantism is merely a matter of convenience, and on-device inference becomes the short road to value.

The consensus is watching the wrong metric

The consensus measures intelligence in parameters and in absolute scores on frontier benchmarks. It is the metric that describes the present accurately and gets the pace of change wrong.

The metric that predicts is a different one: how much capability survives per parameter, once the model is compressed inside a mobile chip. On that ground, the French work offers the first clean number. A small model, trained well, pays less than two points on a standard logical inference task.

Ninety percent of analysts are right about the present. The frontier model leaderboard matters a great deal today, and it will matter little once the enterprise workload shifts to repetitive, verifiable, legally sensitive tasks.

Those tasks live perfectly well inside 0.8 billion parameters: clause extraction, document reconciliation, consistency checking across sources. Frontier cloud remains useful for the long tail of open-ended reasoning.

The curve: three points on the same axis

A trajectory needs at least three points. The paper offers three, all on the same question: how much accuracy it costs to give up raw text and stay inside a readable representation.

• SNLI: 89.7% against the identically trained text model, a gap of 1.9 points.
• ANLI round R2: 48.0% against RoBERTa-large's published 48.9%.
• ANLI round R3: 44.9% against RoBERTa-large's 44.4%.

The fourth point carries more weight than the other three. On round R1 the pipeline loses 16 points, and the overall gap settles between 9 and 14 points.

The authors call that shortfall the price of interpretability, and show that it comes from representational limits rather than from a shortage of data. The distinction matters a great deal: representational limits fall away with better architectures, in months, while data scarcity demands years of collection.

This desk's standing positions set the other axis of the curve: the price of intelligence falls roughly 40-fold a year, and the device beats the cloud because sensors and local inference get cheaper faster than bandwidth does.

The price of interpretability is a line item

Until now, AI auditability lived in compliance slide decks. Now it has a price in hard numbers.

A company choosing a readable pipeline knows exactly what it pays: about two points on simple tasks, up to 14 points on adversarial cases built specifically to break models. That turns a philosophical debate into a line in a contract.

The causal mechanism is clear. A classifier that reads graphs produces an inspectable chain of evidence: atomic proposition, triple, retrieved subgraph. A human reviewer follows the path and points to the step that produced the error.

The opaque cloud model offers the higher score and a closed box. In regulated sectors that box costs fines, expert testimony and lawyer time, items that far exceed the value of two accuracy points.

That is why this measurement is worth more than its academic dimension: it puts a price tag on an architectural choice thousands of CTOs will sign off on over the next twenty-four months.

Cliff event: the 2027-2029 contract renewals

Technology adoption rarely grows smoothly: it jumps, when a constraint falls away.

The constraint here is the audit clause. The moment a small, local, inspectable model comes within two points of the large model, technology buyers gain fresh leverage at the negotiating table.

Cliff event: renewal of enterprise inference contracts in the 2027-2029 cycle, with an auditable local tier as a standard option. The date comes from the cadence of the multi-year contracts signed in 2025 and 2026, which expire inside that window.

Anyone signing a three-year deal today on a cloud-first architecture is buying a dependency destined to age before it expires. It is the classic error of pricing as permanent a scarcity that lasts one cycle.

Silicon is moving with it. Arm is bringing the topic of scaling physical AI to RoboBusiness, a signal that inference at the edge of the network is already a roadmap priority for the people designing the chips, as The Robot Report^[2] reports.

Three categories that will change shape by 2029

The first is low-value inference sold by the token. Vendors living off repetitive classification requests will see that volume migrate onto the customer's device, where marginal cost tends toward zero.

The second is document intelligence software built as a thin layer on top of a frontier API. When the model drops to 0.8 billion parameters and runs inside the customer's perimeter, that layer loses its reason to exist, and proprietary data remains the only defence.

The third is sample-based compliance auditing. A pipeline that exposes the chain of evidence makes total review possible at close to zero cost, and sampling becomes a relic.

Apple, Qualcomm and Arm sit on the right side of this shift. Their customers, meaning enterprise software vendors, will have to rewrite their revenue model before the next funding round.

My position, and what would change my mind

My position, in one line: the value of enterprise AI is migrating toward models small enough to run on a phone, and frontier compute becomes base infrastructure with declining margins.

The reasoning rests on three legs. First: capability per parameter is growing faster than the demand for open-ended reasoning in real-world processes. Second: auditability now has a measured price, and therefore a negotiable one. Third: the cost of shipping data to the cloud stays fixed, while the cost of local inference falls every quarter.

What would change my mind? A capability jump in frontier models that widens the gap back beyond 15 points on common enterprise tasks, combined with a fall in cloud token prices steep enough to make local savings irrelevant.

The gap on ANLI round R1, a full 16 points, deserves respect. It indicates that adversarial reasoning remains the large model's territory, for now.

Prediction, horizon, kill signal

Prediction: by 31 December 2027 at least three of the top ten enterprise generative AI vendors will offer an auditable local inference tier as a standard contractual item, rather than as a pilot project.

Confidence: 68 out of 100. Horizon: 470 days. Kill signal: a recognised public benchmark that, in December 2027, shows sub-2-billion-parameter models more than 10 points below cloud models on standard logical inference tasks.

Inevitable rather than imminent: the direction stays clear, the timing depends on renewal cycles. Whoever signs in 2027 decides where value ends up in 2030.