Blog AI Security

AI Agents on Both Sides: Autonomous Exploitation Meets an Exploited AI Stack

This week's briefing: attacker-run agent frameworks are finding and weaponising bugs on their own, while the AI infrastructure defenders deploy is being exploited and backdoored. Here is what Amazon, Google, CISA and the advisory databases reported, and what it changes for appsec teams.

Published 5 min read
Line-art banner of two mirrored AI agent figures facing each other across a shield, with the headline about AI agents on both sides of appsec

Key takeaways

  • Amazon and Google each documented attacker-run agent frameworks automating vulnerability discovery and exploitation.
  • The agents won on a default-configuration flaw and failed against two other targets, so basic controls still hold.
  • CISA lists a LiteLLM guardrail sandbox escape as exploited, putting LLM gateways in patch scope.
  • Trojanised agent-framework releases on PyPI stole CI and cloud credentials through a .pth startup hook.
  • Apache widened the affected range of an exploited Roller flaw, a reason to re-triage when advisories change.

AI agents are now on both sides of the appsec problem. In one week, two large vendors documented attacker-run agent frameworks doing vulnerability discovery and exploitation without a human choosing each step. In the same week, an LLM gateway landed on CISA's exploited list and Microsoft's agent SDK was trojanised on PyPI. A fifth story, a legacy Java deserialization bug, shows why the boring parts of vulnerability management still decide outcomes.

Attack side: agents that find their own bugs #

Amazon Threat Intelligence reported that an actor ran the open-source Cairn penetration-testing framework against universities in East Asia, giving the agents a list of IP addresses and the goal "obtain shell access". On one target the agents found a previously unknown path traversal in MONAI Label, an open-source medical imaging AI application. Amazon's account: "In about 11 minutes, the agents went from mapping the application's API to arbitrarily writing files on the server." The figure is Amazon's and covers that one stage on that one target.

Two details matter more than the headline number. First, the bug was an unvalidated filename in an upload endpoint, the kind of finding routine code review catches. The maintainer advisory says authentication is disabled by default, so the default configuration was the real exposure. Second, the agents failed elsewhere: Amazon says two other targets absorbed many hours of effort and more than 30,000 credential guesses and held. Amazon attributes the activity to a Chinese-nexus actor with moderate confidence and names no group.

Attack side: patches as input #

Google Threat Intelligence Group and Mandiant say that in the past quarter they "identified threat actors using LLM frameworks to fully automate vulnerability discovery and exploitation without a human in the loop for the first time." On an exposed staging server they found a multi-agent framework dubbed exploit_agent. In Google's description it "compares code before and after patch announcements, analyzes findings to identify potential vulnerabilities, drafts exploits, and automatically executes them without human intervention", and it is set up to run that analysis continuously against popular web application software.

Be precise about what Google shows. It describes how the framework is built and configured. It gives no success rate, no named exploited vulnerability, no victim count and no patch-to-exploit time, and it does not attribute the framework to a named group or country. Google also reports Cairn and a customised platform built on open-source PentAGI in real intrusion activity, and notes that such frameworks "currently trade stealth for velocity".

Patch diffing is how n-day exploits have always been made. Automating it means a public fix commit is itself the disclosure.

Defence side: the AI stack is an attack surface #

CISA added CVE-2026-40217 in BerriAI LiteLLM to its Known Exploited Vulnerabilities catalog. The maintainer advisory, which dates from May, describes a sandbox escape in the proxy's custom-code guardrail test endpoint. That endpoint runs user-supplied Python inside a hand-rolled sandbox that can be escaped, giving code execution in the proxy process, which runs as root in the default Docker image. Reaching the endpoint requires a proxy-admin credential in default configurations, so this is not an unauthenticated bug.

An LLM gateway holds every upstream provider key an organisation owns, so code execution there is a credential-store compromise. The Hacker News, citing Google's report, says attackers used the flaw to escalate limited inference keys to administrative credentials as part of LLMjacking activity. One practical wrinkle: the advisory's text and its metadata name different fixed releases, so upgrade to the later of the two. If you cannot upgrade, the advisory says to block the test endpoint at your reverse proxy or API gateway.

Separately, two malicious releases of Microsoft's agent-framework package reached PyPI. The GitHub advisory says they shipped a credential stealer through a .pth startup hook, which Python runs at every interpreter start, so installing the package was enough. The payload goes after what build systems hold: secrets in environment variables, credential files, Azure managed-identity tokens, and Azure DevOps and GitHub Actions OIDC tokens. How the uploads were made is not confirmed. The advisory's instruction is to treat any environment that installed either release as exposed and rotate the credentials reachable from it.

The reminder: advisory scope changes #

CISA also listed CVE-2026-129448, an unauthenticated deserialization flaw in Apache Roller's XML-RPC endpoint. Apache says it is the same vulnerability as an earlier CVE that named only one affected release, and that it is being actively exploited. Teams on older releases that correctly triaged the first advisory as not affected are now affected. Apache adds that disabling XML-RPC in the admin UI is not enough on older releases.

What this changes #

  • The patch window is shorter. If agents can diff a fix and attempt exploitation unattended, internet-facing apps need faster security update rollout and virtual patching at the edge while rollout happens.
  • Defaults matter more than ever. Authentication off by default exposed MONAI Label to any network client, and the LiteLLM proxy runs as root in its default Docker image, which raises the impact of code execution there. Audit the defaults of everything you deploy, especially research and AI tooling stood up quickly.
  • AI infrastructure belongs in the patch programme. Gateways, agent SDKs and model-serving apps need the same inventory, advisory tracking and upgrade discipline as any production software.
  • CI is the prize. Pin dependencies, add a cooldown before adopting fresh releases, check provenance and scope id-token permissions tightly.
  • Re-triage when advisories change. A closed "not affected" ticket should reopen when a vendor widens the range.

None of this needs a new category of control. It needs the existing ones applied faster and to more of the stack.

Sources

  1. Chinese-nexus actor's Cairn AI agents found a zero-day on their own, AWS Security Blog
  2. Path traversal in datastore upload endpoints allows arbitrary file write, Project MONAI (GitHub Security Advisory)
  3. GTIG AI Threat Tracker: Agentic Threat Activity Crosses a Significant Threshold, Google Threat Intelligence Group
  4. CISA Adds Four Known Exploited Vulnerabilities to Catalog, CISA
  5. LiteLLM has a sandbox escape in custom-code guardrail · CVE-2026-40217 · GitHub Advisory Database · GitHub, GitHub Advisory Database
  6. CISA Warns of 4 Exploited Flaws Linked to Ransomware, Cryptomining, and LLMjacking Attacks, The Hacker News
  7. Malicious code in agent-framework (PyPI) · GHSA-764m-3ppc-5m73 · GitHub Advisory Database · GitHub, GitHub Advisory Database
  8. Microsoft agent-framework PyPI Package Compromised, StepSecurity
  9. CISA Adds Four Known Exploited Vulnerabilities to Catalog, CISA
  10. CVE-2026-129448: Apache Roller: Unauthenticated deserialization in the XML-RPC endpoint (releases before 6.1.5), Apache Software Foundation (Roller user list)

Questions this raises

Did AI agents really find and exploit a zero-day without a human?
Amazon reports that Cairn agents, given a goal and a target list, found and exploited an unknown path traversal in MONAI Label. Amazon assesses with high confidence that the flaw was a zero-day. The same agents failed against two other targets.
Has Google shown AI turning a patch into a working exploit in the wild?
Google describes a framework set up to diff patches, draft exploits and run them automatically. It does not give a success rate, a named exploited vulnerability or a timing.
Is the LiteLLM flaw exploitable without credentials?
Not in default configurations. The maintainer advisory says reaching the vulnerable endpoint requires a proxy-admin credential.
What should an appsec team do first?
Check for the two malicious agent-framework releases and rotate credentials if found, upgrade LiteLLM and Apache Roller, and turn on authentication for any exposed MONAI Label server.

AI Agentssupply-chainCISA KEV