# AI Bans Ineffective in Academic Peer Review, Study Finds Researchers continue using AI tools despite explicit prohibitions in paper evaluations. - **Source:** [New Scientist](https://www.newscientist.com/) - **Published:** 2026-09-28 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/ec3f7e57-fb76-4554-a922-51c6c1538a9b ## Experiment setup at major AI conference Microsoft Research conducted a large randomized experiment during the 2026 International Conference on Machine Learning (ICML) in Seoul, involving 24,661 submitted papers and 17,886 reviewers. Researchers could choose between two review policies: one strictly banning LLM use and another permitting limited AI assistance for understanding papers and polishing writing, but not for making judgments. ## Nearly identical acceptance rates The study found minimal difference in outcomes between the two policies. Papers reviewed under the strict ban had a 27% acceptance rate versus 26.5% under the permissive policy. Average review scores were nearly identical (3.31 vs 3.32 out of 6), and reviewer confidence levels showed no meaningful variation, challenging assumptions about AI's impact on review quality. ## Widespread policy non-compliance An anonymous survey revealed 22.5% of reviewers assigned to the no-AI policy admitted using LLMs anyway. Microsoft's Miro Dudík noted this non-compliance rate exceeded expectations. Violations primarily involved using AI for brainstorming feedback, drafting text, reading papers, and summarizing strengths/weaknesses - tasks explicitly prohibited under the conservative policy. ## Workload drives AI adoption The study identified key reasons for policy violations, including heavy reviewing workloads and unclear rules. University of Wolverhampton's Kayvan Kousha noted the temptation to use AI stems from academic workload pressures. Reviews under the permissive policy were 5.5-7% longer and rated slightly higher quality by experts, though reviewer-by-reviewer comparisons showed no significant differences. ## AI detection challenges Using Pangram AI text detector, researchers found only 52.2% of reviews under the conservative policy were classified as fully human-written, compared to 37% under the permissive policy. However, the team cautioned about the limitations of AI detection tools, suggesting the actual AI usage rates might differ from these measurements due to detection inaccuracies. ## Broader implications for academia Sunnie S.Y. Kim of Microsoft Research believes the findings offer insights for computer science conferences and other academic fields facing similar challenges. The results suggest blanket AI bans may be ineffective, prompting reconsideration of how to responsibly integrate AI tools while maintaining review integrity across scholarly communities. ## Policy design considerations The study highlights the need for clearer guidelines and realistic expectations around AI use in peer review. With nearly a quarter of reviewers ignoring prohibitions, institutions must balance oversight with practical solutions that acknowledge both the benefits and risks of AI assistance in academic evaluation processes. ## Future of academic peer review These findings challenge the assumption that AI bans can be effectively enforced in academic settings. As AI tools become more prevalent, the research suggests academic institutions may need to shift from prohibition to developing frameworks that guide appropriate AI use while maintaining research evaluation standards and transparency. ## Sources covering this story - https://www.newscientist.com/article/2590949-scientists-cant-stop-using-ai-even-when-forbidden-from-doing-so/?utm_campaign=RSS|NSNS&utm_content=home&utm_medium=RSS&utm_source=NSNS — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._