AI security startup Gray Swan has relaunched its red-teaming platform with a redesigned Arena and its largest competition yet, offering $300,000 in prizes to participants who can successfully trick AI agents into violating their safety guardrails.
The Pittsburgh-based company works with frontier AI labs including Anthropic, OpenAI and Google DeepMind to stress-test their models before public release, running an online platform where researchers and hobbyists compete to find vulnerabilities in deployed AI systems.
The flagship new challenge, called Indirect Prompt Injection (IPI) Black, is sponsored by Anthropic, Meta, Google DeepMind and Reflection AI, with new attack scenarios released weekly through November 2.
Indirect prompt injection has emerged as one of the most pressing AI security concerns as companies increasingly let autonomous agents browse the web and take actions on users' behalf. In such an attack, hidden instructions are planted in content the agent encounters during a task, such as a webpage or forum post, in an attempt to hijack its behavior without the user's knowledge.
To keep the testing safe, Gray Swan's Arena runs challenges inside sandboxed environments designed to prevent real-world damage, and competition rules require a human participant to actively drive each AI agent being tested rather than allowing fully automated attacks.
The relaunch reflects a broader shift in AI red-teaming priorities, from testing standalone chatbots for harmful outputs toward probing the more complex failure modes of autonomous agents that can browse, click and execute actions across the open web.
