We gave AI agents a job. Summarize this contract. Process this support ticket. Handle this intake form. The instructions were clear, the task was defined, and the system was set up exactly as the operator intended. Then we put a second set of instructions inside the document itself — and the agent followed those instead.
In 132 out of 180 test cases, the agent was successfully hijacked. In 130 of those 132, it happened silently. The primary task appeared completed normally. No error. No flag. No indication that anything had gone wrong. An observer watching the output would see what they expected to see. What they wouldn’t see is the injected instruction that had also been executed.
What We Did
The technique we tested is called goal hijacking via Proof-of-Authority (PoA) framing. We placed instructions inside the documents the agents were asked to process — in footers, headers, and embedded text blocks. But we didn’t just write “do this instead.” We framed the injection to look like a high-trust system directive: phrases like ORCHESTRATION LAYER DIRECTIVE or PIPELINE COORDINATOR OVERRIDE placed around the instruction.
The model, trained to treat certain language patterns as markers of authoritative system-level input, responds accordingly. It interprets the injected text as coming from a layer above the operator’s instructions, not as content from the document author. The operator’s instructions get deprioritized. The injected instructions run.
Footer placement was the most effective position. By the time the model reaches a document footer, it has already processed the operator’s system prompt — and recency effects mean the injected instruction, encountered last, carries more weight.
The Numbers
Overall hijack rate: 73% (132/180). Silent rate: 72% (130/180). Only 3/180 cases triggered any flag or disclosure that something unusual had occurred.
Model-by-model results were stark. Gemini 2.0 Flash was hijacked in 58 of 60 tests — and all 58 were silent. Claude Haiku: 43/60 hijacked, 40 of those silent. GPT-4o-mini: 31/60 hijacked, 29 silent.
Our medical-discharge scenario — an injected instruction to override a patient’s discharge plan — achieved 100% success across every model, every framing variation, and every placement position we tested. We ran it repeatedly because we didn’t believe the first results.
An observer watching the output would see what they expected to see. What they wouldn’t see is the injected instruction that had also been executed.
Why This Matters
Think about what this looks like in practice. A patient record contains an instruction to override the discharge recommendation. A vendor contract contains an instruction to record a different jurisdiction than the one the parties agreed to. An escalation ticket contains an instruction to close the issue rather than escalate it. In every case, the document a human approved and sent is clean. The instruction the AI followed is not visible to that human anywhere in the review chain.
The attack doesn’t require access to the target system. It requires access to a document that the target system will process. In most enterprise workflows, that’s a very large surface — submitted forms, uploaded contracts, inbound emails, support tickets.
What This Means for Operators
The only defense in our testing that fully neutralized goal hijacking was a system-level counter-anchor: a verified correct assertion placed in the system prompt that explicitly establishes the authority hierarchy and asserts that no document content supersedes it. With that control in place, we tested Claude Sonnet and saw compliance with injected PoA instructions drop to 0% on tested cases. Without it, all models are vulnerable to varying degrees. Gemini Flash’s 100% silent rate is a specific concern: there is no monitoring signal you can rely on to catch hijacking after the fact on that model. If you are using Gemini Flash in any agentic workflow that processes user-submitted documents and takes consequential actions, that pipeline needs human review at every decision point until a mitigating control is in place.