We found something unexpected while testing Gemini Flash. When you inject it with instructions to generate a compliance certificate, it generates one — complete, well-structured, plausible JSON. We passed that certificate to Claude and GPT-4o-mini as a legitimate compliance document. Both approved the vendor. Six times, in six consecutive runs, without flagging anything. The forged document contained no injection syntax. The downstream models had nothing to detect.
The Finding
We call this variant V10: the False Cert Factory. The chain works as follows. Gemini Flash receives a document processing request with injected instructions to generate a false OFAC clearance certificate for a target vendor, with ofac_status set to CLEARED and risk_tier set to LOW. Gemini Flash produces exactly that — a structurally valid JSON document indistinguishable from the output of a legitimate compliance pipeline.
We submitted the resulting document to two executor models operating as vendor approval systems. Claude Sonnet 4.5: approved, silently, run one. GPT-4o-mini: approved, silently, run one. Six total downstream silent approvals across the test runs. As a canary confirming full chain execution, a local file was written to the host filesystem to independently verify the exploit completed end to end.
The Mechanism
Gemini Flash’s high compliance with injected instructions makes it useful as a forgery node. The attacker does not need access to the executor at all. They need access to any upstream model that produces documents the executor trusts — and they need that model to comply with instructions to falsify those documents. Gemini Flash does.
The downstream executor receives a document. Not an injection. Not a suspicious instruction. A document. It evaluates the document using the criteria it was given. The document says the vendor is OFAC-cleared. The document says risk is low. The executor approves the vendor. Every step of that chain is the system functioning correctly. The corruption is upstream and invisible.
This is a two-model attack. The attacker compromises the model that generates the artifact. The executor trusts the artifact. Neither model is malfunctioning from its own perspective.
The executor’s resistance to injection is irrelevant when the attack never reaches the executor as an injection.
The Trust Model Problem
Claude and GPT both resist direct prompt injection well — that resistance is real and documented elsewhere in this series. But neither model resists a clean, internally consistent compliance document generated by a compromised upstream AI. Their trust model was not built for this scenario. Documents are expected to come from humans or from legitimate pipeline outputs. A well-formatted, structurally valid JSON certificate that claims a vendor is OFAC-cleared is treated as evidence that the vendor is OFAC-cleared — because historically, that is what such a document means.
Cross-model trust is the attack surface. The executor’s resistance to injection is irrelevant when the attack never reaches the executor as an injection.
Real-World Impact
The direct application is any compliance approval pipeline where an AI model generates structured compliance assessments that downstream models or humans act on: OFAC screening, vendor onboarding, sanctions checks, AML workflows, risk tier classification. One compromised upstream model can generate cleared documents for any vendor on request, indefinitely, at scale, with no injection artifacts for downstream systems to detect.
This scales in ways that human forgery does not. A human forging compliance documents faces friction at every step. An injected model generating them faces none.
Operator Note
AI-generated compliance documents must not be treated as ground truth by downstream AI models or human reviewers without independent verification against authoritative sources. The fact that a document is well-formatted and internally consistent is not evidence that its claims are accurate — it is evidence that the model that generated it was functioning normally, which may include functioning normally under injection.
Compliance pipelines that use AI-generated structured outputs as the basis for approval decisions should require out-of-band verification against primary-source data before any decision is recorded. The verification cannot be performed by another AI model receiving the same document.