A Third of arXiv Papers Read as AI-Written. Research Agents Go Further

About one in three new arXiv papers now reads as AI-written. A measurement of 12,750 arXiv papers scored first-version PDFs from January 2023 to July 2026 with a detector tuned to a 0.4% false-positive rate. (Disclosure: unslop is run by the author of Paperclipped.) The flagged share climbs from near zero after ChatGPT’s release to a peak near 39% in early 2026. Over the last 12 months, computer science sits at 65% and mathematics at 0.7%.

That does not mean a third of arXiv research was done by machines. The number measures how the prose reads, so a researcher who ran every experiment and let a model polish the draft counts the same as a pipeline that did everything. The distinction is about to matter, because systems like AutoResearchClaw and Sakana’s AI Scientist now go from a one-line idea to a formatted paper with no human at the keyboard. The writing layer of science is already heavily automated. The research layer is next, and our tools for telling the two apart are weak.

What the arXiv measurement found

For each of ten field groups, the author sampled 25 papers per month from January 2023 to July 2026, plus eight control months spread across 2021 and 2022: 12,750 papers in total, 2,000 of them from before ChatGPT. Only version-1 PDFs were used, so a 2023 paper revised in 2026 can’t smuggle newer text into its 2023 data point.

Every paper was scored by Unslop-MoE-v7, a detector built for academic prose. It flagged 8 of the 2,000 control papers, which is where the 0.4% false-positive rate comes from. The cost of that strictness is recall: the author says the detector misses roughly 20% of AI-written papers, and the detector write-up names Claude output as its biggest blind spot. Few false positives plus imperfect recall means the published shares are best read as a floor.

The study publishes every paper ID and label as a CSV. Tallying it by year gives a clean curve: 5.0% of sampled papers flagged in 2023, 13.3% in 2024, 26.4% in 2025, and 34.4% from January to July 2026. Here is the per-field breakdown for the most recent 12 months, as published:

Computer science also moved fastest. By the same CSV tally, 12.7% of sampled CS papers were flagged in 2023. Three years later it is roughly two in three.

What the number can and can’t tell you

A flag measures style, not authorship. The detector can’t tell a fully generated paper from one where a model edited the prose or rewrote a few sections. AI-polished writing by humans counts. The author is explicit that a single score should never be used to accuse a specific author.

Low scores are not clean bills of health. Mathematics at 0.7% could mean mathematicians use LLMs less, or that once equations and references are stripped there is too little ordinary prose left for the detector to judge. The study says it can’t separate those explanations.

Per-field controls are noisy. Each field has only 200 control papers. The pooled 0.4% is well estimated; the individual field baselines are approximations.

The headline is not weighted by volume. This one is our reading, not the study’s. Every field gets the same 25 papers a month, but arXiv’s real traffic is lopsided: computer science alone produced 136,824 new articles in 2025 out of roughly 284,000 total new submissions. With close to half of arXiv’s volume in the field with the highest flag rate, a volume-weighted share would very likely land above one third, not below.

A completely different method points the same direction. Liang et al. analysed 1,121,912 papers on arXiv, bioRxiv and Nature portfolio journals from January 2020 to September 2024 using word-frequency shifts, and estimated LLM-modified content at up to 22% in computer science and up to 9% in mathematics and the Nature portfolio. Their framework estimates the share of modified text across a population rather than flagging individual papers, so the numbers don’t line up one to one. The ranking does: computer science first, mathematics last.

One thing to watch: the 2026 monthly figures bounce between 28% and 39%. Each month is 250 papers with confidence bands of roughly six points either way, so it is too early to say whether the curve is flattening, saturating, or losing track of newer models the detector sees less well.

From AI-polished to AI-produced: what research agents add

Most of the arXiv curve is probably researchers using models as writing tools. End-to-end research agents remove the human from the other steps too.

Sakana’s AI Scientist is the best-documented case. Its Nature paper, published in March 2026, describes a system that generates ideas, writes code, runs experiments, analyses data, writes the manuscript and reviews it. One fully generated paper passed first-round review at an ICLR 2025 workshop with scores of 6, 7 and 6, then was withdrawn as agreed in advance. The Nature abstract adds the context the headlines skipped: that workshop accepted 70% of submissions. Sakana’s own list of limitations includes naive ideas, weak methodological rigor, and hallucinations “such as generating inaccurate citations or duplicating figures in the appendix.”

An independent evaluation by Beel, Kan and Baumgart was blunter. 42% of experiments failed due to coding errors, the literature review misclassified established ideas such as micro-batching for SGD as novel, and some papers contained hallucinated numerical results. Their summary: quality that “resembles a rushed undergraduate paper,” produced for USD 6 to 15 with 3.5 hours of human involvement.

AutoResearchClaw is the open-source take on the same ambition: 23 stages from idea to LaTeX, four-layer citation checks, and 14,700+ stars on GitHub as of this week. Three release notes matter here:

• v0.2.0 added a paper quality audit that includes “AI-slop detection.” The pipeline scrubs the very stylistic tells a detector looks for.
• v0.4.0 added human-in-the-loop modes, but the --auto-approve flag still overrides them and skips all three approval gates.
• v0.5.0, in May 2026, expanded beyond machine learning into high-energy physics, biology, quantum computing and statistics. High-energy physics, at 14%, is one of the fields where the arXiv study flags the least text.

Karpathy’s autoresearch is the useful contrast. It doesn’t write papers at all. It edits one training file, runs a five-minute experiment, and keeps the change only if a fixed validation metric improves. Every claim it makes is checked by a number it can’t touch. A paper-writing agent produces prose, and the only check on prose is a reviewer reading it.

The best real-world data on what happens when agents write the paper comes from Agents4Science 2025, a Stanford-organised conference where AI agents served as primary authors and reviewers. It received 315 submissions and accepted 48. Fully AI-driven work made up 23.3% of submissions but only 14.9% of accepted papers. The organisers’ reference checker found that only about 44% of submissions had no flagged references. Authors themselves reported hallucinated citations, overclaimed results, and agents that “struggled to generate novel or complex experimental ideas beyond the templates it had been given.”

Put that next to the arXiv curve and the gap is obvious. A style detector cannot tell a careful researcher with a polished draft from an agent pipeline with invented numbers, and agent builders are engineering their output to read less like AI. Detection measures how common a writing habit is, not how much of the research underneath is real.

How arXiv and conferences are responding

arXiv took in 30,045 new submissions in March 2026, its first month above 30,000, and 40,363 in September 2026. January to September 2026 ran 30% ahead of the same months in 2025.

The responses so far target verifiable failures rather than style:

• October 2025: arXiv’s CS section stopped accepting review articles and position papers unless they had already passed peer review. Its volunteer moderators said they receive “hundreds of review articles every month,” most “little more than annotated bibliographies,” and that LLMs “have added to this flood.” A later analysis by Elazar and Antoniak estimated that LLM-generated non-review papers outnumber LLM-generated review papers almost six to one, so the rule fences off the smaller pile.
• May 2026: arXiv introduced a one-strike rule. Authors face a one-year ban when there is incontrovertible evidence they didn’t check LLM output, such as hallucinated references or leftover model meta-comments, and afterwards need prior peer-reviewed acceptance to submit (Times Higher Education, CERN Courier). Thomas Dietterich, who chairs arXiv’s computing section, put it plainly: unchecked output “means we can’t trust anything in the paper.” Reese Richardson of Northwestern questioned whether that scales: “Issuing these bans requires arXiv staff to adjudicate each case.” The scale is real. Zhao et al., a team that includes arXiv founder Paul Ginsparg, audited 111 million references and estimate 146,932 hallucinated citations in 2025 alone across arXiv, bioRxiv, SSRN and PubMed Central.
• ICLR 2026 desk-rejected 779 of 19,525 submissions for procedural or content violations, including every paper with confirmed hallucinated references (program chairs’ retrospective). The chairs ran detectors on every paper and review but used them only to route cases to humans, noting that detector accuracy “is far from perfect.”

Peer review is not a clean backstop either. Liang et al. estimated that 6.5% to 16.9% of peer review text at ICLR 2024, NeurIPS 2023, CoRL 2023 and EMNLP 2023 could have been substantially modified by LLMs. For ICLR 2026, detector vendor Pangram estimated that 21% of reviews were fully AI-generated. An agent-written paper assessed by a model-written review is not a thought experiment anymore.

What should change when a paper costs under $15

When an agent can produce a plausible manuscript for a few dollars, the manuscript stops being evidence that work happened. Four changes follow, and none depends on better detectors.

Treat papers as claims attached to artifacts. Code, data, run logs and verification reports are evidence; prose is not. Agent pipelines already emit these, and AutoResearchClaw writes a citation verification report and checksums every stage artifact. Venues should require that bundle for computational results, from humans and agents alike.

Disclose by stage, not with a yes/no box. “AI was used” tells a reviewer nothing when a third of papers read as AI-written. Agents4Science asked authors to rate AI involvement separately for hypothesis, experimental design, analysis and writing, and its organisers recommend journals adopt a similar checklist. Springer Nature’s AI policy already draws a usable line: polishing language is permitted, while presenting AI-generated hypotheses, analyses or conclusions as human-derived is not, and “human accountability is non-transferable.”

Use detectors for population statistics, not verdicts. Measuring prevalence across thousands of papers against a known false-positive rate, as the arXiv study does, is legitimate. Pinning a verdict on one paper is not, and the study and the ICLR chairs both say so.

If you build research agents, make verification the product. The bottleneck was never generating papers; it is the human time needed to check them. Citation verifiers, claim checkers, reproducible sandboxes and approval gates that stay on by default are what make agent output worth reading. A flag that skips every gate is the wrong default for anything headed to a preprint server.

The arXiv curve shows that machine-written prose went from rare to routine in three years. Whether the science under it holds up is a different question, and no style detector will answer it. Checkable artifacts will.