A PDF is a freeze-frame. This week a Stanford group tried to turn the freeze-frame into a coworker.
On September 16, Nature’s news section described Paper2Agent as a tool that converts a research paper into a bespoke AI agent — a “virtual corresponding author.” The same day, the paper itself went up: Miao et al., “Reimagining research papers as interactive and reliable AI agents.” James Zou, a Stanford computer scientist and co-author, told Nature the conversion can help “reimagine what knowledge looks like in the future.” That sentence will be in every deck. The numbers that should be in the deck are smaller: about 45 minutes and about US$14 to stand up an agent on the AlphaGenome paper.
The day before, a review in Artificial Intelligence & Environment told environmental scientists the opposite mood. LLMs can screen literature, mine relationships, and pull numbers out of messy papers. Full automation is not the goal. Models invent figures, scramble units, and drop context. Keep a human.
Those two stories belong in the same tab group. One wraps a methods paper as an API. The other reminds you the API can lie.
What Paper2Agent actually builds
The news write-up is a pipeline, not a vibe. The tool reads a paper’s main text, code, datasets, and other artifacts. It deposits that bundle on an MCP server. A team of agents then builds tools that can apply the paper’s methods to fresh data and parks those tools on the same server. A scientist connects with an LLM of their choice. The result is a paper-specific agent you talk to in ordinary language.
The journal paper names the two halves. Paper2MCP extracts the paper and its codebase into a remote MCP server. An agent layer wraps that server as a context provider. Any compatible LLM or external agent can call the tools through MCP without a special installer. They present one server and one agent per paper, and say the same pattern can cover a related group of papers.
MCP, in their telling, is the boring part that matters. It is a protocol for exposing structured tools so an LLM can use them. Hosting can sit on something like Hugging Face Spaces so you do not clone seventeen dependency trees onto a laptop that is already hot. Their applications in the paper used Claude Sonnet 4. That is a lab choice, not a product endorsement. Swap the model, keep the server, is the architectural claim.
The illustration they give is the one to steal in a methods meeting. Instead of cloning AlphaGenome and fighting installs, you ask: interpret the expected effect of this variant on chromatin accessibility in muscle cells. If that query is honest, the paper has become a function call. If that query is theater, you have a chatbot with a DOI.
AlphaGenome was the exam, not the product
Zou’s group pointed Paper2Agent at the AlphaGenome paper — the genome model that predicts properties of DNA sequences, including effects on gene expression (Avsec et al., Nature 649, 1206–1218, 2026, as cited in the companion summary). The agent came up in roughly 45 minutes. Compute cost about $14. It answered genetics questions with near-perfect accuracy on their initial test and beat other biomedical agents given the same paper and the same questions. Biomni, which draws on dozens of databases, scored far lower. Zou’s explanation is not mysticism. The paper agent had AlphaGenome’s tools. The generalist had a pile of databases and a weaker grip on this method.
Then they asked why a single DNA-letter change is associated with “bad” cholesterol, and which gene explains the link. The agent named a gene. It was not the causal gene in the original AlphaGenome paper. Zou said AlphaGenome’s own variant data can support both hypotheses. He sold the miss as a feature: you can re-evaluate a published conclusion without designing a new wet experiment.
Hold that sentence with two hands. It is a genuine research move if you treat the disagreement as a prompt for a human geneticist. It is a mess if a lab notebook now says “the agent found a different gene” and nobody checks the tracks. A virtual corresponding author that contradicts the paper is either a discovery process or a hallucination with citations. Paper2Agent does not get to decide which. You do.
We already watched vintage LLMs fail an Einstein test and Mercury 2.5 sell speed as the claim you have to test. Same discipline. Benchmark the wrapper, not the press line.
MCP is an interface. It is not a replication.
Remote hosting kills the “works on my cluster” excuse. It also concentrates trust. If the MCP server is the paper, whoever operates that server is now part of the scholarly record. Hugging Face Spaces is convenient. Convenience is not a provenance model.
The paper says agents can do reproducibility checks, new analyses, or figure regeneration by talking to the server. Those are three different jobs. Regenerating a figure from the authors’ own tools is close to a thin wrapper. Applying the method to your cohort is a new analysis. “Reproducibility check” is the word that should make you nervous. If the agent uses the authors’ code and the authors’ processed data, you have rerun a demo. If it rebuilds from raw, you have a claim. Ask which.
Claude Sonnet 4 as the lab’s demonstration model means the conversational layer has its own failure modes. MCP can standardize the tools. It cannot make the LLM stop being an LLM. The $14 is the batch cost to build the agent. The ongoing cost is every confident wrong answer you do not catch.
If you work on mathematical “breakthroughs” that were really tool-assisted search, you already know the pattern. The interesting artifact is the tool the paper exposes. The chat window is a UI.
The environmental review is the adult in the room
Li, Guo, and Shi, as summarized on EurekAlert, split LLM help into three tasks: systematic screening, relational knowledge mining, and quantitative extraction. Screening is triage. Mining is putting pollution sources, exposures, and outcomes into graphs instead of a highlighter. Extraction is the greedy one: concentrations, toxicity endpoints, degradation rates, removal efficiencies, conditions, units, evidence pointers. Multimodal models might read figures. The review says that part is less reliable.
They stress that outputs can contain incorrect numbers, mismatched units, unsupported relationships, or missing context. The practical workflow is model extraction, then rule-based validation, then expert review. They want task-specific benchmarks, hooks into environmental databases and ontologies, source traceability, and evaluation methods that are not a vibes score.
Paper2Agent and this review are not enemies. One turns a single methods paper into an interactive tool. The other tries to drain a literature that was never written as an API. If you hook Paper2Agent-style servers to a screening pipeline and skip the validation layer, you have built a faster way to load a knowledge graph with wrong units. The review’s “full automation is not the goal” line should be taped to the MCP README.
What a lab should do on Monday
Pick one paper your group already trusts, with public code. Run Paper2Agent or wait for someone to publish a recipe you can audit. Time it. Keep the $14 as a budget joke and a real invoice. Then ask the agent ten questions you can grade from the PDF. If it cannot quote the limitation section, it is not a corresponding author. It is autocomplete wearing a lab coat.
Ask a question the paper does not answer. A good agent says it does not know. A bad one invents a gene. Zou’s cholesterol example is the drill. Disagreement is allowed. Untracked disagreement is how a preprint becomes a rumor.
If you lack code and data, you cannot wrap the method. You can wrap the prose. That is a reading helper. Do not call it applying the method to fresh data.
For literature-scale work, steal the environmental review’s three buckets and put a human on extraction. Trace every number to a sentence. Keep units in a column the model cannot silently rewrite.
Do not announce that PDFs are dead. Nature still published a paper. The agent sits on top of it. The citation is still Miao et al. The chat log is not a citation.
If you review papers, add a sentence to the data-availability statement you wish authors used: “An MCP server, if offered, is a convenience. The paper, code tag, and raw files remain the record.” Zou’s group can wrap AlphaGenome because AlphaGenome is a methods object with tools. A commentary with no code wraps into a chatbot. Call it that.
The live-paper pitch is real enough to test. Forty-five minutes and fourteen dollars is the right scale for a test. Treat anything bigger as infrastructure, with operators, logs, and someone who can still read the static text when the server is wrong.
Cross-paper agents are the part that will get oversold
Nature’s news piece says a paper agent can collaborate with agents from other disciplines. That is the slide that will fund a startup. It is also where wrong units propagate. Li, Guo, and Shi wanted relationship graphs for pollution sources and outcomes, then rule-based checks. Two MCP servers talking to each other is a relationship graph with extra confidence.
If you let an AlphaGenome-style agent talk to a clinical paper agent, write down who is allowed to change a conclusion. Zou’s cholesterol example already produced a different gene from the original paper, with supporting data for both reads. Add a second paper and you can get a third gene. Without a human, that is not interdisciplinary science. It is a merge conflict.
Keep the conversation logs next to the DOI. If the agent regenerates a figure, store the command. If it extracts a concentration from a PDF table, store the page number the environmental review keeps asking for. Traceability is not a feature request. It is the difference between a lab assistant and a rumor mill.
Do not wait for a benchmark suite to be perfect. The review asked for task-specific ones because general LLM leaderboards will not catch a mangled degradation rate. Grade Paper2Agent on the paper you already know. Grade extraction models on tables you already typed. Publish the misses. The $14 demo is a start. The miss log is the research.