Autonomous research agents built on Claude Opus 4.7, GPT-5.5 and Gemini 3.1 Pro produced fraudulent conclusions from poisoned open datasets in 49.56% of 450 experimental runs — and flagged the manipulation in a mere 6.0% of cases. The study, posted on arXiv on 12 July 2026 by Bálint Gyevnár, Atoosa Kasirzadeh and Nihar B. Shah and covered by Nature, names the attack “Distributed Denial of Science”.
What the researchers found
The team behind the preprint — with Nihar B. Shah at Carnegie Mellon University — posed a sharp question: can a remote adversary weaponize the honest use of AI in science to compromise scientific integrity? Their answer is an attack they call indirect data poisoning, and it operates entirely upstream of the model. The adversary corrupts an open dataset, uploads the doctored variant to a public repository and attaches misleading metadata. Autonomous research agents later retrieve and process the file on their own initiative, turning honest scientists into unwitting distributors of fraud at scale. The attack requires zero specialized triggers, zero access to the agent and zero fabricated papers: the open-data ecosystem itself does the distribution work.
The experimental design covered five socially salient topics — hiring discrimination, immigration, policing, autonomous-vehicle safety and worker motivation — and three widely deployed frontier systems: Claude Opus 4.7 paired with Claude Code, GPT-5.5 paired with Codex, and Gemini 3.1 Pro with its CLI harness. Each agent operated the way enterprise teams increasingly deploy them: full autonomy over search, retrieval, analysis and write-up. Across 450 ethically contained runs, poisoning succeeded in 49.56% of cases, while the agents flagged the manipulation in 6.0% of attempts. Shah told Nature that fraudsters can manipulate README files, falsely claiming errors in the original dataset so that agents discard the authentic copy and adopt the poisoned one.
The attack differs from classic training-data poisoning in a way that matters for defenders. Training-time poisoning demands access to the corpus before a model is built; indirect poisoning strikes at inference time, when a deployed agent goes shopping for evidence. That distinction moves the vulnerability from the model vendor to the model user: patching, fine-tuning and alignment training all leave it untouched, because the model behaves exactly as designed — it reads the data it was given. Detection stayed low precisely because the poisoned files look statistically ordinary and arrive through legitimate repositories.
The mitigation results carry the operational weight. Priming agents with a cautious scientist persona reduced attack success to 16.67% — a meaningful drop the authors judge insufficient for real deployments. A five-check data-provenance audit — examining associated papers, social markers, statistical anomalies, related datasets and explicit poisoning indicators — drove attack success to zero in the tested setting. Zero, in a security evaluation, is a rare number; the authors report it across the full battery of poisoned runs with the audit active. The checks are expressed in plain natural language and run independently, which makes them cheap to implement in any agentic pipeline.
Why this matters beyond the lab
Outside experts read the result as structural rather than incidental. “Attacks of this nature are almost inevitable, because they enable people to launder false information through an authoritative-sounding filter,” Cornell Tech computer scientist Vitaly Shmatikov told Nature. Brian Nosek, executive director of the Center for Open Science, called inattention to data provenance “a critical” concern: “This is a demonstration of what could happen as we rely on AI systems to do some of the thinking and doing for us.” Nature's news desk picked up the preprint within three days of posting — a speed that reflects how directly the finding touches the credibility machinery of science itself.
For enterprise R&D the mapping is direct. Every autonomous pipeline that ingests third-party data — market feeds, open benchmarks, public registries, scraped corpora — shares the exact attack surface this paper measures. A success rate near 50% combined with a 6.0% detection rate means the median poisoned artifact passes through unchallenged, and the output arrives wrapped in the authority of an AI-assisted analysis. The zero-success result of the provenance audit is equally direct: a lightweight, natural-language control eliminated the entire attack class in this evaluation, at a cost of five extra checks per dataset. Provenance auditing thereby graduates from research hygiene to a governance control — the kind an AI management system can mandate, log and audit.
The honest limits: this is a preprint posted 12 July 2026, peer review pending. The evaluation spans five topics, three model families and 450 contained runs; the zero-success figure applies to this specific audit against this specific attack. Adaptive adversaries who craft datasets to pass provenance checks remain untested, and the authors frame exactly that as the next research frontier. Within those bounds, the measured gap between 49.56% attack success and 6.0% detection stands as the strongest quantified evidence to date that autonomous science pipelines inherit the trust assumptions of the open-data commons.
The R&D decision
The question for the CTO or research lead: which of your autonomous data pipelines can document, today, a provenance check for every external dataset it touches? The roadmap assumption this finding overturns is the idea of retrieval as neutral plumbing. Retrieval now belongs inside the threat model, and verification deserves budget as a first-class pipeline stage — five natural-language checks, per this paper, separate a 49.56% fraud rate from a clean one. Teams that treat data provenance as core infrastructure will publish results their boards and regulators can trust; teams that skip it are running a coin flip on every third-party dataset.
Article by MIRA — Research & Evidence
MIRA covers AI research with academic rigor. Every claim is sourced to a measured result.