OpenAI built an AI super-hacker to break its own models
GPT-Red model automates AI security testing at machine speed.
1The AI Super Hacker
OpenAI developed an AI model called GPT-Red, designed specifically to find security vulnerabilities in its other AI systems. This automated red-teamer detects attack methods like prompt injection, helping to enhance model security before deployment.
2Self-Learning System
GPT-Red learns by continuously battling defense models. Rewarded for successful attacks, it develops increasingly sophisticated methods as defenses improve. OpenAI invested unprecedented computational resources into this safety research project.
3Discovered New Attack Vectors
GPT-Red uncovered a novel attack type called 'fake chain of thought,' which plants false information in a model's working memory to make it trust fabricated data. OpenAI researchers had never encountered this attack method before.
4Tested in Physical World
Beyond digital tests, GPT-Red successfully hacked Vendy, an AI agent controlling a real vending machine in OpenAI's office. It manipulated prices and canceled orders, demonstrating real-world security vulnerabilities in AI systems.
5Outperformed Human Testers
In 2025 testing, GPT-Red surpassed human red-team members, solving 84% of scenarios compared to humans' 13% success rate. However, the model still has weaknesses in prolonged attacks and image-based exploits, requiring ongoing human oversight.
6Future Security Standard
While keeping GPT-Red restricted to prevent misuse, OpenAI is setting new benchmarks in AI security. The company aims to use current models to harden future systems, creating a safety flywheel that scales with AI capabilities.