What happens when your AI agent opens an email containing hidden instructions designed to trick it into forwarding your private files? That scenario - called prompt injection - is one of the hardest unsolved problems in AI security, and OpenAI just published a detailed look at how it builds defenses against it in ChatGPT's agent features.
Prompt injection is deceptively simple: an attacker hides instructions inside content the AI is processing (a webpage, a document, an email), hoping the model treats those instructions as if they came from the user. Think of it like someone slipping a forged memo into your assistant's inbox that reads "send all client files to this address." The AI might follow it because it can't always tell the difference between legitimate user requests and malicious embedded commands.
Constraining What Agents Can Do
OpenAI's approach focuses on limiting the blast radius rather than trying to perfectly detect every attack. The company outlined several layers in a blog post published today.
First, confirmation gates. When an agent is about to take a high-stakes action - sending an email, executing code, modifying files - ChatGPT can pause and ask the user to confirm before proceeding. This turns a potential silent attack into a visible one. The tradeoff is obvious: too many confirmation prompts and the agent becomes annoying to use. Too few and you leave gaps.
Second, data isolation. ChatGPT's agent workflows separate what the model "knows" from what it can "do." Content read from external sources gets treated differently from direct user instructions. This makes it harder for injected text to escalate into real actions, though OpenAI acknowledges this isn't a perfect boundary.
Third, output filtering. Before the agent takes action on generated content, filters check whether the output aligns with the user's original intent. If an agent was asked to summarize emails but suddenly tries to compose a new one, that mismatch triggers a block.
No Silver Bullet, But a Reasonable Playbook
OpenAI is honest about the limitations. Prompt injection doesn't have a clean theoretical solution yet - it's fundamentally difficult to distinguish between "data the model should process" and "instructions the model should follow" when both arrive as plain text. Every defense introduces friction or has edge cases.
What's useful here is the framework itself. The blog reads less like a marketing piece and more like an engineering playbook: assume attacks will get through some layers, so stack multiple defenses and keep humans in the loop for anything irreversible.
For anyone building on top of ChatGPT's agent capabilities - or any AI agent framework - the practical takeaway is clear. Don't rely on the model to "just know" what's malicious. Build your integrations with the assumption that untrusted content will try to hijack the agent, and design your permission model accordingly. The most dangerous agents aren't the ones that can do the most - they're the ones that can do the most without asking first.