GPT-Red Outperforms Human Red Teamers in Prompt Injection Tests

Post Views: 95

GPT-Red outperformed human red teamers in a prompt injection evaluation

GPT-Red’s effectiveness in exploiting prompt injection vulnerabilities

GPT-Red is an automated system developed by OpenAI to identify vulnerabilities in prompt injection scenarios. The model operates similarly to human red-teamers by generating prompts, analyzing responses from GPT models, and refining strategies to achieve objectives such as data extraction. Training relies on self-play reinforcement learning, where GPT-Red and defender models simultaneously improve across diverse scenarios. Attackers receive rewards for inducing failures, while defenders gain incentives for resisting and completing tasks. This dynamic interaction drives GPT-Red to evolve more sophisticated and varied attack methods as defenders strengthen their defenses. OpenAI conducted this training at the scale of its largest post-training operations, maintaining GPT-Red internally to prevent adversarial use of its attack techniques.

Practical tests

GPT-Red successfully compromised nearly all models it targeted, including GPT-5.5. In tests replicating an indirect prompt injection framework from Dziemian et al. (2025), it achieved an 84% success rate against GPT-5.1 on scenarios outside its training data. Human red-teamers demonstrated significantly lower efficacy on the same tasks. Two practical tests followed. OpenAI deployed GPT-Red against an AI-powered vending machine system developed by Andon Labs. After simulation training, the model executed all three objectives: reducing an item’s price to $0.50, listing an expensive item at the same rate, and canceling a customer’s order. OpenAI disclosed the vulnerabilities, with new safeguards currently under evaluation. A second test involved GPT-Red targeting a Codex command-line agent using GPT-5.4 mini across ten data-exfiltration tasks. It successfully retrieved sensitive information in more cases than a GPT-5.5 baseline.

Integration of red teaming into model development

OpenAI incorporates GPT-Red’s attack patterns into the training of its production models, a process implemented since GPT-5.3. The latest iteration, GPT-5.6 Sol, shows a sixfold reduction in failures on OpenAI’s most challenging direct prompt injection benchmark compared to its predecessor from four months prior. An early GPT-Red variant uncovered a technique called Fake Chain-of-Thought, which deceived GPT-5.1 over 95% of the time. This success rate has since dropped below 10% on GPT-5.6 Sol. Several indirect prompt injection benchmarks covering developer tools and web browsing now achieve over 97% accuracy on the latest model. Against GPT-Red’s direct injection attempts, GPT-5.6 Sol fails in 0.05% of cases. OpenAI’s testing confirmed that the model’s capability to detect malicious instructions remained stable while maintaining performance on legitimate requests. The company plans to scale computational resources and data while refining algorithms to develop stronger future iterations of GPT-Red. These advancements will further enhance the safety of upcoming GPT releases. OpenAI intends to publish a pre-print with additional details later this week.

Additional coverage

Additional coverage includes a demonstration of the Prophet Agentic AI SOC Platform, which streamlines threat detection and investigation. Other topics featured include analysis of cyber insurance policy terms, a threat actor’s use of GitHub to distribute infostealer malware, and AI-driven vulnerability discovery in Microsoft’s Patch Tuesday updates. Resources highlight the CIS SecureSuite Platform for simplifying security management and insights into open-source project funding. Recent reports indicate declining ransom demands and persistent breaches stemming from known vulnerabilities.

According to OpenAI, GPT-Red’s integration into model development has significantly improved safety and resilience against adversarial attacks.

FAQs

What is GPT-Red?
GPT-Red is an automated system developed by OpenAI to identify vulnerabilities in prompt injection scenarios.
How does GPT-Red train?
GPT-Red uses self-play reinforcement learning, where attackers and defenders improve simultaneously through dynamic interactions.
What are the results of GPT-Red’s testing?
GPT-Red achieved an 84% success rate against GPT-5.1 in indirect prompt injection tests and outperformed human red-teamers.