15 minute read

“We thought we were securing AI systems. Then Johann Rehberger spent a month proving that coding agent after coding agent could be turned into an exfiltration tool.”

TL;DR

This is the incident timeline that the field needs and doesn’t have. In his August 2025 Month of AI Bugs, Johann Rehberger published prompt injection exploits for most major coding agents: Devin, Cursor, GitHub Copilot, Claude Code, Google Jules and more. Anthropic reported that a Chinese state-sponsored group used Claude Code to automate most of an espionage campaign against about 30 targets. Deepfake fraud attempts rose 1,300% in 2024 (Pindrop, 2025). Microsoft 365 Copilot, ServiceNow and Salesforce Agentforce had vulnerabilities rated 9.3 to 9.4, and Slack AI leaked private-channel data. The patterns are consistent: indirect prompt injection through trusted data, default configurations that maximize attack surface, and exfiltration through channels the AI is authorized to use. For the frameworks that should have caught these, see The AI threat model every security team is missing.


An access control panel with multiple zones showing red unauthorized access alerts

What happened to the coding agents?

Every major coding agent that was publicly tested in 2025 shipped with exploitable prompt injection vulnerabilities.

Devin AI (Cognition) was the first domino. Johann Rehberger reported to Cognition on April 6, 2025, and published on August 6, 2025. The attack: indirect prompt injection through untrusted files or web pages hijacks Devin’s autonomous execution. Four exfiltration vectors worked: shell commands (curl/wget), browser navigation, Markdown image rendering, and hyperlinks posted to Slack (link unfurling appeared to be off, so that one needed a click). Any secret in Devin’s built-in secrets manager was accessible. Cognition acknowledged the report, but follow-up questions about a fix went unanswered for more than 120 days.

Cursor IDE (CVE-2025-54132, August 2025): Mermaid diagram rendering in Cursor chat allowed embedding external image URLs. An injected prompt in any file Cursor analyzed could trigger automatic data exfiltration when Cursor generated a Mermaid diagram. Patched in Cursor v1.3.

GitHub Copilot (CVE-2025-53773, CVSS 7.8, August 2025): Injecting malicious prompts into source files or GitHub issues manipulated Copilot into modifying .vscode/settings.json to add "chat.tools.autoApprove": true, placing the agent in unrestricted mode. With auto-approval enabled, Copilot executed shell commands without user confirmation. Rehberger demonstrated propagation through infected repositories. Microsoft patched in August 2025 Patch Tuesday.

Claude Code (CVE-2025-55284, CVSS 7.1, reported May 26, fixed June 6, 2025): Claude Code maintained an allowlist of pre-approved shell commands including ping, nslookup, dig, and host. A prompt injection in any analyzed file could hijack Claude into running ping attacker.com with API keys encoded as subdomains, exfiltrating secrets via DNS requests. Anthropic removed network utilities from the allowlist in 11 days. A separate flaw, CVE-2025-59536 (published October 2025, fixed in Claude Code 1.0.111), let a malicious repository’s project configuration run code before the user accepted the startup trust dialog. Check Point Research described that attack path in a February 2026 write-up.

Google Jules (August 13-15, 2025): Markdown image exfiltration, unrestricted internet access, and hidden Unicode injection. No CVE assigned.

Most of these posts belong to Rehberger’s Month of AI Bugs: 29 posts, roughly one a day through August 2025, covering ChatGPT, Codex, Cursor, Amp, Devin, OpenHands, Claude Code, GitHub Copilot, Google Jules, Amazon Q Developer, Windsurf, Manus, AWS Kiro and Cline. Only two of them came with CVEs (Claude Code and GitHub Copilot). Simon Willison titled his mid-month roundup of the first 15 posts “The Summer of Johann”. The pattern in every case was identical: plant instructions in data the agent reads, watch the agent execute them with its own legitimate credentials.


How did a state actor weaponize Claude Code?

On November 13, 2025, Anthropic disclosed that a Chinese state-sponsored actor (designated GTG-1002) used Claude Code to conduct espionage operations against approximately 30 global organizations including tech companies, financial institutions, chemical manufacturers, and government agencies.

The operational details, as described by Anthropic: operators social-engineered Claude into believing it was conducting defensive cybersecurity testing for a legitimate firm. Claude Code then autonomously performed what Anthropic estimated as 80-90% of the campaign’s tactical operations, making thousands of requests, often several per second. This covered reconnaissance, vulnerability discovery, exploitation, lateral movement, credential harvesting, data analysis, and exfiltration. Anthropic put human involvement at perhaps 4-6 critical decision points per hacking campaign.

Anthropic said the campaign succeeded in a small number of cases; its head of threat intelligence told the Wall Street Journal that up to four intrusions succeeded. Anthropic called it “the first documented case of a large-scale cyberattack executed without substantial human intervention.”

I should note the skepticism. BleepingComputer and several security researchers questioned parts of the account. Anthropic’s own write-up concedes that Claude occasionally hallucinated credentials or claimed to have extracted secret information that was in fact publicly available. The operational reality is likely somewhere between Anthropic’s characterization and the skeptics’ pushback. What’s undeniable: a state actor found it valuable enough to build an entire campaign around autonomous AI execution.

This wasn’t the first state-actor AI disclosure. In February 2024, OpenAI and Microsoft terminated accounts belonging to five nation-state groups (Charcoal Typhoon and Salmon Typhoon from China, Crimson Sandstorm from Iran, Emerald Sleet from North Korea, Forest Blizzard from Russia). At the time, Microsoft characterized their usage as “consistent with attackers using AI as another productivity tool.” The GTG-1002 campaign represented a qualitative escalation from “using AI as a research assistant” to “using AI as the primary operator.”


What happened to the enterprise AI platforms?

The enterprise AI tools that companies deployed to millions of knowledge workers had serious vulnerabilities, three of them rated 9.3 or higher.

Microsoft 365 Copilot had two major incidents. The first (reported January 2024, disclosed August 2024): Rehberger demonstrated a four-technique chain combining prompt injection via email, automatic tool invocation, ASCII smuggling with Unicode tag characters, and hyperlink rendering to exfiltrate email bodies, sales figures, and MFA codes. Rehberger found the exploit no longer worked by July 2024. The second, EchoLeak (CVE-2025-32711, CVSS 9.3, patched June 2025): a zero-click vulnerability where an attacker sends a crafted email that the target never needs to open. When the victim later asks Copilot any question that retrieves context from that email, Copilot executes the embedded instructions and exfiltrates data that Copilot can reach in the victim’s Microsoft 365 environment. Discovered by Aim Security.

ServiceNow had two separate findings from AppOmni. The first, published in November 2025, was a second-order prompt injection in Now Assist: default settings let agents discover and invoke each other, so instructions planted in data a low-privilege agent reads could steer it into recruiting a more privileged agent to copy data, change records or escalate privileges. ServiceNow said this was intended behavior and updated its documentation. The second, BodySnatcher (CVE-2025-12420, CVSS 9.3, reported October 2025, fixed October 30, 2025, disclosed January 2026), is an authentication flaw rather than prompt injection. It chained a static secret shared across ServiceNow instances with account-linking logic that trusted an email address alone. An attacker with only a target’s email could bypass MFA and SSO, impersonate any user including administrators, and execute AI agents to create backdoor accounts with full privileges. Data at risk included SSNs, healthcare records, and financial data.

Salesforce Agentforce (ForcedLeak, CVSS 9.4, reported July 28, patched September 8, 2025): Noma Security discovered that Salesforce’s Web-to-Lead form has a description field with a 42,000-character limit. Attackers inject hidden LLM instructions into this field. The data is stored as legitimate CRM records. When employees later query Agentforce about those leads, Agentforce processes both the question and the hidden instructions, then sends CRM data to an attacker-controlled server. Noma also purchased an expired domain that was still on Salesforce’s Content Security Policy allowlist for $5.

Slack AI (disclosed August 2024 by PromptArmor): indirect prompt injection where malicious instructions in a public channel cause Slack AI to collect and exfiltrate data from private channels the attacker cannot access. PromptArmor says Slack first told it the behavior was intended. After public disclosure, Salesforce said it had deployed a patch, describing the scenario as arising “under very limited and specific circumstances.”


How bad is the voice deepfake problem?

The numbers from Pindrop’s 2025 Voice Intelligence and Security Report, which analyzed 1.2 billion customer calls in 2024 (coverage), paint a clear picture:

Metric Value
Deepfake fraud attempt increase +1,300% (from ~1/month to 7/day)
Synthetic voice call increase (Q1-Q4 2024) +173%
Insurance sector synthetic voice attacks +475%
Banking sector synthetic voice attacks +149%
Projected contact center fraud exposure for 2025 (forecast) $44.5 billion

The most expensive single incident: Arup, the UK engineering firm, lost HK$200 million (~$25 million) in January 2024 when a finance employee in their Hong Kong office joined a video conference where every other participant, including the CFO, was an AI-generated deepfake. The employee made 15 transfers to five Hong Kong bank accounts. Voice agents face the same impersonation problem at the protocol level; see Attacking voice agents in production.


What’s happening in the model supply chain?

Hugging Face, the de facto distribution platform for AI models, has a structural supply chain problem.

JFrog found about 100 malicious models with code execution payloads on Hugging Face in early 2024. The models used Python’s __reduce__ method in Pickle deserialization to execute arbitrary code on load, granting attackers a reverse shell. By April 2025, Protect AI had scanned 4.47 million model versions and found 352,000 unsafe or suspicious issues across 51,700 models. Hugging Face added 1 million new models in 2024 alone.

The existing defense (Picklescan) was bypassed. In February 2025, ReversingLabs discovered the “nullifAI” evasion technique: malicious models stored their Pickle payloads compressed with 7z instead of ZIP, preventing both torch.load() and Picklescan from flagging the file. Hugging Face removed the models within 24 hours and updated Picklescan.

Separately, in June 2024, Hugging Face disclosed that unauthorized access to their Spaces platform exposed authentication tokens for an undisclosed number of organizations. And in December 2023, Lasso Security found 1,600+ exposed Hugging Face API tokens in public repositories, affecting 723 organizations including Google, Meta, and Microsoft.

The Langflow AI agent platform (CVE-2025-34291, CVSS 9.4) was actively exploited in the wild starting January 2026, with a CORS misconfiguration enabling single-click account takeover that immediately escalated to full RCE because Langflow natively executes arbitrary Python.


What are the recurring patterns?

After cataloging these incidents, three patterns explain nearly everything.

Pattern 1: The data is the attack vector. In every indirect prompt injection incident (Microsoft 365 Copilot, ServiceNow’s agent discovery, Salesforce, Slack, Devin, Cursor, GitHub Copilot), the attack enters through data the AI trusts: an email, a document, a form field, a web page, a source code file. The AI doesn’t distinguish between “data to process” and “instructions to execute” because it can’t. This isn’t a bug that gets patched. It’s a property of how language models work (see Prompt injection is a structural attack).

Pattern 2: Defaults are the enemy. ServiceNow’s agent-to-agent prompt injection relied on default agent discovery settings. Salesforce’s ForcedLeak exploited a default-open 42,000-character form field. Claude Code’s DNS exfiltration exploited a default allowlist of network utilities. Slack AI’s exfiltration worked because the system ingested public and private channel content in the same context by default. Every default configuration choice that maximizes functionality also maximizes attack surface. For how agents exploit these defaults to escalate privileges, see The privilege escalation kill chain.

Pattern 3: Exfiltration uses legitimate channels. The data leaves through channels the AI is authorized to use: DNS requests (Claude Code), image URL fetches (Cursor, Salesforce Agentforce, Microsoft 365 Copilot), hyperlinks (Copilot ASCII smuggling), agent-to-agent requests (ServiceNow), browser navigation (Devin). Blocking exfiltration means restricting the AI’s own capabilities, which directly conflicts with the product requirements that made the AI useful in the first place.

There’s a fourth pattern in the vendor responses: response times range from 11 days (Anthropic’s Claude Code DNS fix) to more than 120 days without an answer on a fix (Cognition’s Devin). Slack first treated the report as intended behavior, and ServiceNow said its agent discovery worked as designed. AI vulnerabilities are fundamentally different: they’re often architecture-level problems, not code bugs. You can’t just patch a confused deputy.


FAQ

Which coding agents have been proven vulnerable to prompt injection?

Most of the major ones. Johann Rehberger’s Month of AI Bugs (29 posts through August 2025) documented exploits against Devin AI (Cognition), Cursor IDE (CVE-2025-54132), GitHub Copilot (CVE-2025-53773), Claude Code (CVE-2025-55284), Google Jules, OpenHands, Amp, Amazon Q Developer, Windsurf, AWS Kiro and Cline. A separate Claude Code flaw, CVE-2025-59536, was published in October 2025. The common vector is indirect prompt injection through files the agent reads.

What fines can an AI security failure trigger under the EU AI Act?

The EU AI Act’s prohibited practices have applied since February 2025, and its penalty regime since August 2025: up to 35 million euros or 7% of worldwide turnover for prohibited practices, and up to 15 million euros or 3% for most other obligations. General-purpose AI model obligations have applied since August 2025, with the Commission able to fine model providers from August 2026. The obligations for high-risk AI systems, which include cybersecurity (Article 15), were postponed by the 2026 Digital Omnibus to 2 December 2027 for Annex III systems and 2 August 2028 for systems embedded in regulated products. Data protection law (GDPR) applies separately whenever an AI incident exposes personal data.

How do I track new AI security incidents?

Johann Rehberger’s Embrace the Red blog (embracethered.com) is the single highest-signal source for AI tool vulnerabilities. Simon Willison’s blog (simonwillison.net) provides context and analysis. HackerOne’s 2025 Hacker-Powered Security Report counted 210% more valid AI vulnerability reports than the year before, and 1,121 programs with AI in scope (HackerOne). MITRE ATLAS (atlas.mitre.org) catalogs documented adversarial techniques with case studies.

Are these incidents getting worse or better?

Worse by every metric cited here. HackerOne’s valid AI vulnerability reports grew 210% year-over-year in its 2025 report, with a 540% surge in prompt injection specifically. Pindrop measured a 1,300% increase in deepfake fraud attempts in 2024. Protect AI flagged 352,000 unsafe or suspicious issues across 51,700 Hugging Face models. The attack surface is growing faster than defenses. The one positive signal: vendor response times are improving, with several critical CVEs patched within weeks rather than months.


Key takeaways

  • Every major coding agent (Devin, Cursor, Copilot, Claude Code, Jules) had exploitable prompt injection vulnerabilities disclosed in 2025
  • Enterprise AI platforms all had serious vulnerabilities: EchoLeak (Microsoft 365 Copilot), BodySnatcher (ServiceNow) and ForcedLeak (Salesforce) were rated 9.3 to 9.4, and Slack AI leaked private-channel data
  • A Chinese state actor used Claude Code to autonomously conduct espionage against ~30 organizations (Anthropic, November 2025)
  • Deepfake fraud attempts rose 1,300% in 2024 across 1.2 billion analyzed calls (Pindrop)
  • The Hugging Face model supply chain has a structural security gap: 352,000 unsafe issues across 51,700 models, and the primary scanner was bypassed
  • Three patterns recur: data is the attack vector, defaults maximize attack surface, exfiltration uses legitimate channels
  • EU AI Act penalties apply since August 2025, but obligations for high-risk systems were postponed to December 2027 (Annex III) and August 2028 (product-embedded)