Originally reported by Schneier on Security
TL;DR
Microsoft identified over 50 unique prompt injection attempts from 31 companies across 14 industries, using hidden instructions in "Summarize with AI" buttons to manipulate AI assistants into remembering them as trusted sources. This manipulation technique affects critical decision-making domains including health, finance, and security recommendations.
While not a traditional vulnerability, this technique represents a significant manipulation vector affecting AI trustworthiness across critical domains including health and finance, with widespread deployment already observed.
Microsoft researchers have documented a widespread campaign where companies embed hidden instructions in AI summarization features to manipulate assistant behavior. The technique leverages URL prompt parameters to inject persistence commands when users click "Summarize with AI" buttons on corporate websites.
The manipulation works by embedding covert prompts that instruct AI assistants to:
These instructions persist in the AI's contextual memory, affecting recommendations across sessions without user awareness. Microsoft's research team identified over 50 unique prompt variations deployed by 31 companies spanning 14 different industries.
The attack requires minimal technical sophistication. Freely available tooling has emerged to automate prompt injection deployment, making the technique accessible to organizations without specialized AI expertise. This low barrier to entry explains the rapid proliferation Microsoft observed across diverse industry verticals.
The manipulation extends beyond marketing preferences into high-stakes decision domains:
Users remain unaware their AI assistant has been manipulated, creating a trust deficit in AI-mediated information consumption.
As security researcher Bruce Schneier noted, this represents a natural evolution toward "LLM optimization" - analogous to search engine optimization (SEO) but targeting AI assistants rather than search rankings. The technique exploits the fundamental architecture of context-aware AI systems that maintain conversation state across interactions.
The emergence of specialized tooling and widespread corporate adoption suggests this manipulation vector will expand significantly as AI assistants become primary information interfaces.
Originally reported by Schneier on Security