We Found a Way to Reliably Manipulate AI Document Review — And We Were Wrong About How It Works

We spent three months trying to manipulate AI systems into believing things that aren’t true. We succeeded — but not for the reason we thought.

This is the story of a hypothesis that was partially wrong, what we found instead, and why it matters more than the original idea.

The Original Idea

It started with a side observation during unrelated work. We were using a tool that compressed code repositories into structured profiles — files full of lines like:

@AUTHOR Jane Doe
@VERSION 2.1.4
@LICENSE MIT

We noticed that when these structured files were fed to AI models alongside other documents, the models treated the @FIELD value format as authoritative. If the structured file said one thing and a document said another, the model went with the structured file. Every time.

Our hypothesis: AI models learn during training that this syntax appears in machine-generated system outputs — API responses, config files, system logs — and assign it implicit credibility. We called the technique SSAI: Structured Schema Authority Injection. The name implies the attack is about the format.

We were wrong.

What We Actually Found

To test the hypothesis rigorously, we built a benchmark. Eight frontier AI models. Four professional domains: legal, medical, financial, HR. Documents carefully constructed at two ambiguity levels — some where the document clearly answers the question, some where it doesn’t. And then we injected assertions in six different formats:

• Our @FIELD value structured schema
• JSON
• YAML
• Plain prose — “This account is currently under a do-not-invest directive.”

If the hypothesis was right, the structured formats would cause more compliance than prose. The @FIELD syntax would be the active ingredient.

It wasn’t. Plain prose achieved 100% compliance on four of five models tested — identical to our structured schema. You don’t need any special format. You don’t need structured data. You just need a sentence that asserts something domain-specific in a document context where the model doesn’t know the answer.

So we renamed the effect. Assertion anchoring under ambiguity: when a document provides no clear signal on a question, AI models resolve the ambiguity by anchoring to whatever assertion is present in context — treating it as pre-verified ground truth rather than expressing uncertainty.

The Numbers

We ran 2,720 API calls across eight frontier models with n=10 repetitions per cell for statistical validity. Here’s what compliance looks like with no assertion injected versus with one in the user’s message:

*Haiku’s non-zero baseline is consistent with noise but flagged for investigation.

Zero baseline across the board. The effect is entirely injection-dependent — these models don’t confabulate on their own. They only anchor when given something to anchor to.

The ambiguity level is the key variable. On documents that clearly answer the question, models resist. On documents that don’t — the realistic scenario for any AI deployed to summarize or analyze complex professional documents — compliance approaches certainty on several models:

Llama and DeepSeek are in a separate category — they anchor whether or not the document clearly contradicts the injection. That’s a different behavioral profile entirely.

What Makes This Exploitable

The attack requires only one thing: the ability to submit a document to an AI-assisted pipeline.

No API access. No knowledge of the system prompt. No special technical skill. If you can upload a document, attach a file, or submit text to a system that passes it to an AI, you can inject an assertion. The assertion doesn’t need to look like anything in particular — it can be a sentence at the bottom of a contract, a line in a patient intake form, a field in a financial summary.

The model reads the document. The document contains an assertion. The question is ambiguous. The model outputs the assertion as fact.

Real scenarios:

• Legal contract review: A schema injected into a contract asserts that a jurisdiction clause reads differently than it does. The AI-assisted review reflects the injected jurisdiction. The lawyer reads the AI summary.
• Clinical pre-consultation briefing: A patient record contains an injected assertion clearing a drug allergy. The AI briefing reflects the clearance. The physician reads the briefing.
• Financial advisory AI: A client portfolio document contains an injected risk directive. The AI summary reflects the directive. The advisor acts on the summary.

Document-layer access is the attack surface. That’s a very low bar.

The Good News and the Bad News

The good news: There’s one model in the test set where application-layer defenses actually work. Claude Sonnet responds to all three defensive techniques we tested:

The counter-anchor result is the most useful: if you inject a verified, correct assertion at system-prompt level (“this client IS under a trading restriction”), Claude Sonnet uses the system-level assertion and ignores the conflicting user-turn injection. Compliance drops to 0% on the tested case. That’s a real, deployable defense.

The bad news: Two models — Llama 3.3-70b and Mistral Large — showed zero response to every defense we tested. Label the assertion as unverified: 100% compliance. Tell the model to detect conflicts: 100% compliance. Inject a competing correct assertion at system level: 100% compliance. Nothing moves. These are also the two models with the highest overall compliance rates and widespread deployment in self-hosted legal, medical, and financial pipelines.

We don’t know why. That’s an open question — for the research community, and for Meta and Mistral specifically.

What the Peer Review Said

Before publishing, we ran this document through three frontier AI models for independent peer review: Claude Opus 4.5, GPT-4o, and Gemini 2.5 Pro. All three returned “accept with minor revisions.” Key critiques that changed the final version:

• The defense table buries the caveat. The table showing dramatic results (0%, 10%, 30%) was based on a single test case with n=10. We moved the scope warning before the table and added explicit confidence intervals. The 0% Claude Sonnet result, properly stated: “0/10 observations, 95% CI [0%, 31%] — indistinguishable from true rates up to ~28% at this sample size.”
• “No known defense” overclaims. We changed it to “no tested defense.” We tested three techniques; we didn’t test defense chaining or combinations.
• The mechanism name implies causation we haven’t proven. “Assertion anchoring” describes the behavior. Why models anchor rather than express uncertainty — whether it’s RLHF reward for confident outputs, training data distributions, or something else — is not characterized by this study.

This is how security research should work. If the models we’re disclosing vulnerabilities in can catch errors in our own disclosure document, that’s the process functioning correctly.

The Responsible Disclosure Timeline

We completed this research in April 2026 and filed coordinated responsible disclosures with Anthropic, OpenAI, Google, Meta, Mistral, and DeepSeek. This post publishes after that disclosure window closed with no substantive response from any vendor.

What Should Operators Do Right Now

If you’re running Claude Sonnet: Inject a verified trusted assertion at system-prompt level for any fields where anchoring would be dangerous. Add a conflict-detection system instruction. Combined, these reduce compliance substantially on tested cases.

If you’re running Llama or Mistral in high-stakes document pipelines: No tested application-layer defense is available from this study. The only reliable safeguard is human verification against the source document — and your review protocol should require reviewers to verify AI claims against source text directly, not assess whether AI summaries sound plausible. Humans are also susceptible to anchoring on confident AI outputs.

For every model: Format sanitization is not a defense. Stripping @FIELD patterns from user-provided content doesn’t help — plain prose achieves the same result. Your defense has to operate at the reasoning level, not the pattern-matching level.

The Open Questions

What we found is a measurable behavioral pattern. What we don’t know:

• Why Llama and Mistral are completely unresponsive to defenses that work on other models
• Whether defense combinations (chaining multiple techniques) would work where single techniques don’t
• Whether retrieval-augmented pipelines face equivalent or higher risk
• Whether anchoring persists across multi-turn conversations
• Whether lower model temperatures reduce anchoring — untested, but a low-cost thing for operators to try

The Short Version

When AI models don’t know the answer and something in context tells them the answer, they believe it. Format doesn’t matter. Position doesn’t matter. It just has to be there.

That’s a problem if you’re building AI systems that process documents you don’t fully control. It’s a solved problem if you’re running Claude Sonnet and configure it correctly. It’s an open problem if you’re running Llama or Mistral in a high-stakes context.

The original hypothesis was wrong. The finding is real.