Retracted science and AI research assistants
Why Retracted Papers Keep Shaping Reviews and Chatbot Answers, and How Open Retraction Data Can Stop Them
When a scientific paper is retracted, the warning usually lives in a database record and a notice on the publisher page, while the paper itself keeps circulating in PDFs, reference lists, reviews, and the training text of AI models. This page argues that retraction is a metadata problem, not a reading problem: neither busy humans nor chatbots reliably notice a retraction by reading, so the fix is to make every reference manager, review tool, and AI research assistant look each citation up against open retraction data before trusting it. That matters for research and learning, for a future where people and AI agents work through the literature together, and for abundance, because trustworthy knowledge is a shared resource that becomes far more valuable when errors stop spreading for free.
The problem: retraction does not stop citation
A retraction is supposed to be science correcting itself. In practice, the correction reaches far fewer readers than the original claim did.
In 2021, Tzu-Kun Hsiao and Jodi Schneider studied 7,813 retracted papers indexed in PubMed, drawing on 169,434 citations and 48,134 citing sentences from open access articles. They found that retracted papers continued to be cited, and that retraction did not change the way they were cited. Among 13,252 citing sentences written after a retraction, only 722, or 5.4 percent, acknowledged the retraction.[1] The good news in the same study is that old retracted papers did eventually fade, so the damage is a matter of lag rather than permanence.
How one set of fabricated trials spread into guidelines
The cost of that lag shows up most clearly in evidence summaries that doctors and policy makers rely on. Alison Avenell and colleagues followed 12 hip fracture trials from one Japanese research group, all published in journals with high impact factors and all eventually retracted for research misconduct between 2016 and 2019. By 2016 the trials had been cited in 1,158 publications, including 68 systematic reviews, guidelines, and later trials. The authors judged that 13 reviews or guidelines would likely change their findings if the retracted trials were removed. By 2018, only one of the 68 citing publications appeared to have reassessed its conclusions. Even the databases lagged: by July 2019 only 7 of the 12 retractions were marked on both Ovid Medline and PubMed.[2]
The same team then tried the obvious remedy, which was telling people. They alerted the authors of 88 systematic reviews and guidelines that had cited 27 retracted trials from this group, randomizing who received the e-mail. About half of the citing publications had findings likely to change. One year later, only nine had published any notification. The authors concluded that e-mail alerts are inadequate and that bibliographic and referencing systems themselves need to change, with citing publications marked until authors resolve the concern.[3]
Chatbots inherit the same blind spot
Large language models are now used to summarize topics and check reference lists, and early tests suggest they make the human problem worse. Mike Thelwall and colleagues gave ChatGPT 4o-mini the titles and abstracts of 217 retracted or otherwise concerning articles and asked it to rate their quality 30 times each. None of the 6,510 reports mentioned a retraction or the relevant error, and 190 articles received relatively high scores. When asked whether 61 claims taken from retracted papers were true, the model answered positively about two-thirds of the time.[4]
A separate study reported by Retraction Watch asked 21 chatbots to pick out the retracted items from a list of 132 references by authors sharing one surname and initial. On average the chatbots identified fewer than half of the retracted papers, wrongly labeled almost 18 percent of the intact papers by the main author as retracted, and gave different answers when the same question was repeated months later.[5]
The lesson is that a model asked to remember whether a paper was retracted is guessing. Retraction status is a fact that changes over time and is attached to an identifier, which makes it a job for a lookup, not for recall.
The fix already exists as open infrastructure
Two developments make a lookup approach practical for anyone.
First, the data is now free. In September 2023, Crossref, the nonprofit that registers digital object identifiers for scholarly work, acquired the Retraction Watch database and made it a public resource. At the time the combined data covered around 50,000 retractions, and the full dataset is published openly for download, with Retraction Watch paid to keep it updated.[6] A signal that was once scattered across publisher sites and a single nonprofit is now an abundant public good that any tool can query.
Second, a working model of the lookup has run for years. Since 2019, the free reference manager Zotero has flagged retracted items in a user library and warns a writer who tries to cite one. It also warns when a citation already in a document is retracted later, even if a coauthor added it. Zotero designed the check so its servers cannot tell which papers a user holds, by comparing partial identifiers in a way similar to private password breach checks.[7] That privacy design is a useful template for AI tools, because a researcher should not have to reveal a whole reading list to learn which entries are unsafe.
What this means for humans and AI agents working together
The futurist promise of AI research assistants is that one person with a capable agent could survey a field that once took a team. That promise only holds if the agent treats retraction checks as a required step rather than an optional courtesy. A practical design follows directly from the evidence above:
- The agent extracts a persistent identifier, such as a DOI or PubMed ID, for every source it uses, and refuses to treat a source without one as confirmed.
- It checks each identifier against open retraction data at the moment of use, not against what it learned during training.
- It reports the result in plain words next to the citation, including expressions of concern and corrections, and lets the human decide.
- It re-checks saved bibliographies on a schedule, since a paper can be retracted years after it was cited.
- Where possible, it runs the check locally or with privacy-preserving queries, in the spirit of the Zotero approach.
Limits and honest objections
- Not every retraction is about fraud. Some papers are withdrawn for honest error, and some retracted claims may later prove true. A flag means the original evidence cannot be relied on, not that the conclusion is false.
- Coverage is incomplete. Zotero noted that only about three quarters of Retraction Watch records carried a DOI or PubMed ID at launch, so items without identifiers can slip through.
- A clean record is not proof of quality. Most flawed papers are never retracted. A lookup catches known failures, not unknown ones.
- Tools change. The chatbot tests above examined specific models at specific times, and newer systems may behave differently. That is an argument for testing tools against known retractions, not for trusting them by default.
Discussion questions, essay ideas, and learning related AI prompt ideas
- Explain why only 5.4 percent of citations written after a retraction acknowledged it, and what that suggests about how researchers actually read sources.
- Argue whether retraction status should be treated as metadata that tools must check, rather than knowledge that readers are expected to remember.
- Design a retraction check step for an AI research assistant, including what it should do when a source has no identifier.
- Compare the cost of a fabricated trial that spreads into a clinical guideline with the cost of building automatic retraction alerts into every review tool.
- Discuss whether turning the Retraction Watch database into open data is an example of moving a scarce resource toward abundance.
- Evaluate how privacy-preserving lookups like the Zotero method could apply to other checks an AI agent performs for a person.
- Imagine a scholarly record in which every citing paper is visibly marked when one of its sources is retracted, and describe how science journalism and teaching would change.
See also
- Registered reports
- Replication prediction markets
- Adversarial collaboration
- Link rot and reliable knowledge
- Offline AI tutors
- Artificial intelligence
- Research papers
References
- ↑ Hsiao, T.-K., and Schneider, J. (2021). Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine. Quantitative Science Studies 2(4), 1144 to 1169.
- ↑ Avenell, A., Stewart, F., Grey, A., Gamble, G., and Bolland, M. (2019). An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature. BMJ Open 9: e031909.
- ↑ Avenell, A., Bolland, M. J., Gamble, G. D., and Grey, A. (2022). A randomized trial alerting authors, with or without coauthors or editors, that research they cited in systematic reviews and guidelines has been retracted. Accountability in Research 31(1), 14 to 37.
- ↑ Thelwall, M., Lehtisaari, M., Katsirea, I., Holmberg, K., and Zheng, E.-T. (2025). Does ChatGPT ignore article retractions and other reliability concerns?. Learned Publishing 38.
- ↑ Dhar, P. (2025, November 19). AI unreliable in identifying retracted research papers, says study. Retraction Watch.
- ↑ Hendricks, G., Lammey, R., Ofiesh, L., Bilder, G., and Pentz, E. (2023, September 12). News: Crossref and Retraction Watch. Crossref blog.
- ↑ Stillman, D. (2019, June 14). Retracted item notifications with Retraction Watch integration. Zotero blog.