# Detecting AI-Generated Content in Training Datasets Researchers develop methods to identify and measure AI-modified text in peer reviews and datasets. - **Source:** [Towards Data Science](https://towardsdatascience.com) - **Published:** 2026-09-26 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/26996d67-a28a-4ec3-8b45-f9678de3ef3f ## AI-modified peer reviews A 2024 study found that 6.5% to 16.9% of peer review text in major AI conferences showed signs of substantial AI modification after ChatGPT's launch. These reviews were written by researchers making technical judgments with real submission consequences, highlighting how AI-generated content has entered critical academic evaluation processes. ## Model collapse risk A separate 2024 Nature paper demonstrated that training AI models on recursively generated data can cause them to lose rare examples from original datasets, producing narrower results. This phenomenon, called model collapse, shows the long-term risks of using AI-generated content in training pipelines without proper detection mechanisms. ## Detection methodology Researchers developed three metrics to identify AI-generated text: perplexity (predictability of wording), near duplicate similarity (comparison with other reviews), and embedding density (closeness to nearest neighbors). These methods were tested on a dataset combining 200 human-written movie reviews with 400 AI-generated ones. ## Perplexity limitations While perplexity (measuring how predictable text is to a language model) caught 80% of AI-generated reviews, it also flagged 98.5% of human-written references. This high false positive rate makes perplexity alone insufficient for reliable detection, as human writing can also be highly predictable in certain contexts. ## Embedding density results Embedding density, which measures how closely a review resembles its five nearest neighbors, performed better than perplexity but still flagged 47% of human-written reviews when set to catch 80% of AI-generated ones. This shows AI and human writing can occupy similar semantic spaces in certain datasets. ## Combined score performance Averaging all three detection methods resulted in flagging 73% of human-written reviews to catch 80% of AI-generated ones - worse than using embedding density alone. This suggests simple combination approaches may amplify rather than mitigate individual methods' weaknesses in current implementations. ## Real-world implications When applied to 594 archive reviews not used in training, the combined method flagged 444 as potentially AI-written. However, the same method marked 79% of verified human-written test reviews as suspicious, demonstrating current techniques' impractical false positive rates for real-world application. ## Case study: Amadeus review The highest-scoring human-written review was a factual summary of the film Amadeus, scoring 0.796 (above the 0.548 cutoff). Its compact, reference-style writing triggered detection algorithms despite being human-authored, showing how content style rather than origin drives current detection metrics. ## False negative example An AI-generated review about 'The Last Stop on Neptune' scored 0.384 (below cutoff), escaping detection despite containing obviously artificial elements like repetitive plot devices and unnatural dialogue. This illustrates how some AI content can evade current detection methods while human writing gets flagged. ## Impact on model performance Filtering out flagged reviews (removing 466 of 600) reduced sentiment classification accuracy from 67.5% to 57.5%. Downweighting suspicious reviews also hurt performance, suggesting current detection methods may remove valuable human-written data while retaining some AI-generated content. ## Dataset considerations The study used 1,000 movie reviews published on Mendeley in 2019 (pre-ChatGPT), treating them as human-written based on source and date. However, the dataset doesn't verify individual authorship, highlighting challenges in obtaining reliable ground truth for AI detection research. ## Future detection challenges The results demonstrate that current inexpensive detection methods create unacceptable tradeoffs between catching AI content and preserving human writing. As AI generation improves, distinguishing artificial text will require more sophisticated approaches that understand content meaning rather than just statistical patterns. ## Sources covering this story - https://towardsdatascience.com/ai-slop-is-now-in-your-training-dataset-i-tested-three-ways-to-spot-it/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._