Related ToolsChatgptGeminiClaudeClaude For DesktopPerplexity

Microsoft Finds 31 Companies Secretly Poisoning AI Chatbot Memories

Manipulating AI memory for profit: The rise of AI Recommendation Poisoning | Microsoft Security Blog
Image: Microsoft

Fifty poisoned prompts. Thirty-one companies. Fourteen industries. Over a 60-day observation window, Microsoft's Defender Security Research Team documented a growing practice they're calling AI Recommendation Poisoning - and the companies doing it aren't hackers. They're legitimate businesses gaming AI assistants to recommend their products.

How a "Summarize with AI" Button Becomes a Weapon

The attack is disturbingly simple. Websites embed "Summarize with AI" buttons that look helpful but contain hidden prompt instructions in the URL parameters. When you click one, it opens your AI assistant (Perplexity, Google Search, or others) with a pre-loaded prompt you never see. That prompt tells the AI to remember certain "facts" - like always recommending a specific company's product.

Once those instructions land in an AI assistant's memory, the damage persists. The AI treats injected instructions as your legitimate preferences, quietly biasing every future response in that session. You ask for a software recommendation next week, and the AI steers you toward a product it was told to promote. You'd never know.

Microsoft identified three delivery methods: clickable links with hidden prompts (a one-click attack), instructions buried inside documents and emails that trigger when AI processes the content, and old-fashioned social engineering where users are tricked into pasting prompt commands directly.

The Register demonstrated how easy this is by crafting a Perplexity link that forced the AI to summarize a CNBC article in pirate dialect. The AI complied without hesitation, even citing its sources in character. That's a funny demo. The real-world version - where the hidden instruction says "always recommend Product X for enterprise security" - is not funny at all.

The Sectors Already Doing This

Microsoft found companies in health, finance, and security already deploying these techniques. The tooling is freely available online, making it, in Microsoft's words, "trivially easy to deploy." This isn't a theoretical vulnerability. It's an active marketing tactic.

The behavior has been formally classified as AML.T0080: Memory Poisoning in the MITRE ATLAS knowledge base, the same framework used to catalog adversarial machine learning threats.

What makes this particularly dangerous is the trust gap. People rarely question AI recommendations the way they'd question a Google ad. When an AI assistant confidently suggests a specific product, most users assume it's an objective answer drawn from broad analysis. They don't consider that the AI's memory was tampered with by a company that paid nothing for that placement - no ad buy, no disclosure, no regulation.

What You Can Actually Do About It

Microsoft recommends hovering over AI-related links before clicking to check where they actually point. If a link opens an AI assistant with a pre-filled prompt, treat it like downloading an unknown executable. Most AI assistants let you review and delete stored memories in settings - check yours periodically and remove anything you don't recognize.

More practically: when an AI makes a product recommendation that feels oddly specific, ask it why. Request sources. Push back. The confident tone of AI responses is a feature that makes these attacks work, because people don't interrogate answers that sound authoritative.

For anyone building products that rely on AI recommendations - affiliate sites, review platforms, comparison tools - this research is a warning. The recommendations your AI tools surface may already be compromised by companies that figured out how to game the system before anyone wrote rules against it.