An LLM Drops Hardware Specs Even When the Whole Datasheet Fits in Context

Haotian Qiao and Robert P. Dick had a large language model read 20 hardware datasheets in two different ways. Staged reading reaches 98.8% property faithfulness against 76.1% for single-pass reading^[1], according to the paper submitted to arXiv on October 8, 2026.

According to the authors, single-pass reading loses specifications even when the document fits entirely within the text the model can read at once.

Twenty datasheets and two ways of reading them

A datasheet is the technical document for an electronic component. It contains the specifications needed to select the part and to check that it works with the others. The authors describe long documents that differ widely from one another, with data scattered across tables, figures and text.

The sample comprises 20 datasheets covering different components and lengths. The figures in this article come from the paper's public abstract, which does not say which model was used.

Both methods produce a knowledge graph that organizes the properties of each component.

Between 76.1% and 98.8% lie 22.7 points of faithfulness

Faithfulness measures how many extracted properties match what the datasheet actually says.

With single-pass reading, 76.1% of the properties are faithful to the document. The remaining 23.9% were judged unfaithful to the source.

The second figure concerns coverage. The staged graph fully covers 83.6% of the faithful properties found by single-pass reading. In the opposite direction, single-pass reading fully covers 10.5% of the properties in the staged graph.

The three design choices the authors tie the result to

The authors attribute the result to three design choices.

• Text per query: the pipeline splits the document into chunks and reduces how much the model reads on each request.
• Data provenance: each property stays linked to the point in the datasheet it comes from.
• Graph edits: the model writes instructions that are checked, then a program executes them deterministically on the local machine.

The properties are then sorted into a hierarchy and reviewed over several rounds. The abstract presents these choices together and does not say how much each one contributes.

A 2023 study had already observed a related limitation. In Lost in the Middle^[2], models answered better when the relevant information sat at the beginning or the end of a long text. That study tested multi-document question answering and key-value retrieval, so it concerns tasks different from datasheets.

The two shares the better method leaves uncovered

Two shares fall outside the better result. 1.2% of the properties in the staged graph are unfaithful to the datasheet. 16.4% of the faithful properties from single-pass reading are covered partially or not at all by the staged graph.

The staged method therefore covers incompletely some specifications that single-pass reading had found.

Completeness is measured between two graphs

The authors call their measure cross-graph completeness. It compares how much one graph covers the other. There is no exhaustive list of the true specifications of each datasheet, as the authors themselves state.

The 83.6% therefore says how far the staged method outperforms the other. How many datasheet specifications are missing from both graphs remains outside the measure.

Then there is the question of who judges faithfulness. G-Eval^[3], a 2023 study on summaries and dialogues, used a language model as the judge and found higher agreement with human judgments than earlier metrics. The same authors noted that a judge of this kind can favor texts written by a model.

Qiao and Dick's abstract does not say what kind of judge was used for faithfulness.

Which technology thesis these numbers support

The data support a specific thesis. On 20 datasheets, staged reading yields more faithful properties and more coverage than single-pass reading.

The thesis that a larger context is enough to read a technical document well comes out of this sample diminished.

The authors write that the method reduces the risk of overlooked constraints in design. The abstract gives measurements on the graphs and does not report on design errors avoided.

An investment committee can read the 98.8% as a measure of faithfulness on 20 documents from one domain. The share of specifications absent from both graphs is the figure that is missing before completeness can be discussed.