New Method Protects Kids from Illegal AI-Generated Content
MIT and Thorn develop a technique to detect harmful AI models.
1The Dark Side of AI Models
While generative AI models are used for beneficial purposes like creating artistic content, they can also be adapted by malicious actors to produce illegal content such as child sexual abuse material (CSAM). This poses a serious threat, especially to children's safety.
2Rise in CSAM Reports
In 2025, the National Center for Missing and Exploited Children received over 1.5 million reports of AI-generated CSAM cases. This number is significantly higher than the 67,000 reported in 2024, highlighting the scale of the problem.
3Limitations of Traditional Methods
Testing AI models for their potential to produce harmful content typically involves examining their outputs. However, generating CSAM, which is illegal, makes this testing process impossible, underscoring the need for a new approach.
4Collaboration Between MIT and Thorn
MIT and child safety organization Thorn have developed a new technique to detect harmful AI models. This technique analyzes the model's internal structure to determine its potential to produce harmful content without generating any output.
5Gaussian Probing Technique
Researchers use a technique called Gaussian probing to analyze changes within the model. This method can accurately identify models with the potential to produce harmful content with 100% accuracy, helping platforms enhance security.
6A Significant Step for Child Safety
This technique offers a new way to measure the potential of AI models to produce harmful content. It represents a major step forward in child safety and fills a critical gap in AI security.
Original article: MIT News — AI — Yeni Yöntem Çocukları Yasa Dışı AI İçeriklerinden Koruyor