Incident Response Playbooks for Compromised AI Agents

A security team discovers that a prompt injection attack has been running undetected against a production AI assistant for three weeks. Every instinct trained into that team by a decade of ransomware and business email compromise response points them toward the wrong first move, because the playbook they're running was built for systems that don't behave anything like the one in front of them. Agents break all three assumptions at once, and the four failure modes that result, semantic opacity, credential amplification, persistent memory poisoning, and machine-speed damage, demand a response built from the ground up.

Traditional IR playbooks against a compromised AI agent

A compromised server behaves predictably. A compromised agent behaves in natural language, and a poisoned instruction reads no differently from a legitimate one at the level where most security tools actually look. The Cloud Security Alliance frames this precisely: an attacker manipulating a chatbot is, from the network's perspective, just another API call returning a 200 response. The compromise lives entirely in meaning, not in form.

That single fact dismantles the patchability assumption that most IR frameworks build on. There is no binary to patch when the attack surface is a sequence of words crafted to redirect an agent's intent. Signature-based detection, the foundation of most SOC tooling, cannot distinguish a poisoned instruction from a legitimate one because both are valid language, both are valid API calls, and both can produce a valid, successful response. You have known-bad patterns to match against, but here there are none.

The evidence assumption fails for a related reason. Standard event logs record that a user requested a file deletion, not the context window that produced the decision to delete it. The full prompt chain, the sequence of inputs, retrieved context, and intermediate reasoning that led an agent to act, is the only artifact that reveals whether the agent was manipulated or simply wrong. That artifact is typically absent from the logs most organizations already collect. The evidence responders need rarely exists at the moment they need it.

Four failure modes follow from these broken assumptions, and each one is categorically distinct from anything a server-era or account-era playbook was built to handle. Persistent memory poisoning means a compromised agent may have written corrupted data into RAG indexes, vector databases, and shared context stores, and unlike a compromised server that gets reimaged clean, restarting a poisoned agent simply reloads the poisoned context. Machine-speed damage means the agent can act as fast as its rate limits and permissions allow, because no human typing speed is there to slow it down.

The governance gap compounds all four. Most organizations deploying agents in production have not extended their incident response frameworks to cover AI-specific scenarios. No one has acknowledged the mismatch exists yet, so this becomes the condition that lets a compromise go undetected for three weeks.

What constitutes an AI security incident

An AI security incident is any event in which the confidentiality, integrity, or availability of an AI system, or of the data and tools that system can reach, is compromised through attack vectors specific to AI that traditional security systems cannot identify. That definition matters because it draws a boundary traditional monitoring simply does not see. The Cloud Security Alliance identifies five recurring incident types: prompt injection, data poisoning, model poisoning and supply-chain attack, tool poisoning and misuse, and rogue agent deployment. None of these necessarily involves malware, a compromised credential, or a network intrusion, the three things a traditional SOC is built to detect. An incident can unfold entirely within this blind spot while every conventional alert stays quiet.

A published escalation taxonomy gives this problem a shape responders can actually use. The taxonomy starts at Level 1, direct prompt injection against passive outputs. At Level 2 comes jailbreaking at the session level. Level 3 is indirect prompt injection that produces persistent-state changes, so the damage can outlive the session that caused it. Level 4 is agentic exploitation of tool chains and MCP servers, producing cascades that cross system boundaries. Level 5 is still emerging: it is exploitation of trust between agents, and it produces cascades that cross organizational boundaries. Each level implies a different scope of investigation and a different set of systems that need to be checked.

Before you take any containment action, triage has to classify which of five distinct compromise types has occurred, because each one demands a different response path. Supply-chain poisoning means a tool, an MCP server, or a model the agent depends on was tampered with upstream, the pattern you see in a documented technique where malicious models uploaded to a public model-sharing platform hid a reverse shell inside a Python pickle file and evaded the platform's scanner in the process. Lateral propagation means the compromise has moved across agent-to-agent communication channels, and peer agents that were never directly attacked get reached by it.

Severity has to account for what was reachable from the compromised session, not only for what the attacker actually did with it. The OWASP Agentic Skills incident response framework sets four severity bands with response-time targets attached: critical incidents, meaning active exploitation with confirmed data exfiltration, carry a one-hour response window; high severity, a confirmed malicious skill, carries four hours; medium severity, suspicious behavior, carries one business day; low severity, a minor finding, carries one week. Those targets only function if an organization has already decided who owns an AI incident before one occurs, whether that's the SOC, the application team, the AI platform team, or a cross-functional group formed specifically for this purpose. Classification determines not just what playbook to run but who is authorized to run it, and getting that wrong costs exactly the hours the severity bands above assume are available.

You treat that as a credential incident alone and miss the agent-amplification dimension sitting on top of it, and that classification error is what the next section examines directly.

Detecting a compromised agent

Detecting a compromised AI agent takes substantially longer than detecting a compromised infrastructure account because the signals involved are behavioral and semantic rather than binary, and because most organizations are not yet logging the artifacts that would close that gap. An account takeover produces discrete, checkable facts: a login from an unfamiliar location, a password reset, a new device fingerprint. An agent compromise produces a pattern of decisions that look reasonable on their own and only turn suspicious in aggregate, so detection depends on baselines and correlation.

Speed makes this gap dangerous. The fastest attacks now reach data exfiltration in roughly 72 minutes, and in at least one documented case exfiltration began just four minutes after initial access. Against that pace, a detection gap measured in days guarantees that the response arrives after the damage is already done.

Step Finance illustrates exactly this dynamic. In January 2026, attackers compromised executive devices, and they used that access to reach AI trading agents that held permissions to execute large transfers of SOL tokens. The agent's permissions turned a device compromise into a treasury-draining event at a speed no human operator could have matched by hand.

So closing that gap starts with logging a baseline that looks nothing like standard infrastructure logging. Agents need records of every prompt submitted, every output generated, every tool and function call made, every query issued to the retrieval system, and the identity context behind each of those interactions. Without that record, there is no way to establish what happened, when it started, or how far it spread, because the standard event log was never built to capture intent, only action. Tracing needs to go down to the level of individual tool-call arguments, recording the tool name, the arguments passed, where each argument's value came from, the tool's output, any errors, and the downstream action taken, because tool misuse usually originates in an unsafe or untrusted parameter value. This creates a real tension with privacy obligations, since prompts routinely carry personal and confidential data, and the answer is to treat these logs as sensitive archives with strict access controls, bounded retention, and redaction built in, not to log less.

An agent running several concurrent sessions across different tasks, each with its own tool connections and its own MCP server sessions, gives you an anomaly surface far wider than a single-session account compromise ever generates, and that is why detection takes longer if you don't already have the logging baseline above in place before the incident starts.

Containment steps that are specific to agents, and the single most common mistake that makes a containment fail

The single most common mistake in agent incident response is restarting the compromised agent and treating the restart as remediation. It is an understandable instinct, because restarting a compromised server or forcing a password reset on a compromised account genuinely does clear the threat in those contexts. An agent is different: if its persistent memory, its RAG index, its vector store, or its shared context has been poisoned, restarting the agent simply reloads the poisoned context, and the agent resumes operating under the attacker's instructions starting from its first action after the restart completes. The compromise was never in the running process. It was in the data the process reads on every cycle, and a restart does nothing to that data.

Correct containment has to undo a deeper assumption carried over from server-era playbooks, the assumption that isolating the affected session isolates the problem. An agent identity often runs several concurrent sessions, each with its own tool connections, so if you revoke only the session that triggered the alert, every other session keeps running under that same compromised credential. The OWASP Top 10 for LLM Applications 2025 names Excessive Agency as a specific failure mode here, caused by excessive functionality, excessive permissions, or excessive autonomy, and containment has to address that root cause by reducing the agent's scope.

Three containment actions need to happen in parallel, not in sequence. Credentials have to be revoked across every connected system immediately, not just on the session that triggered the alert. The agent has to be isolated from any inter-agent communication channel to stop lateral propagation to peer agents before it starts. State has to be snapshotted for forensics before any remediation step touches persistent stores, because the memory and context an agent carries are the primary evidence of how it was manipulated, and that evidence is destroyed the moment sanitization begins.

Containment has to be time-boxed because damage scales directly with how long the compromised agent stays active, which puts the response window at minutes, not hours. High-confidence actions with a small blast radius, revoking a session tied to a confirmed credential compromise, isolating a single agent, can execute immediately, with no need to wait on approval chains built for slower incidents. Actions that cross systems, segmenting a network, revoking credentials en masse, pulling a third-party token, still require a human approver, but the agent itself should prepare the action and assemble the supporting evidence in advance, so that approval takes seconds.

Reconstructing the semantic chain in forensic investigation of an agent compromise

Blast radius assessment for a compromised agent has to answer a harder question than what the attacker did. It has to answer what was reachable from the compromised session, and answering that requires reconstructing the full semantic chain of inputs, context, and tool calls rather than reading through an event log line by line. Investigators need to establish which documents were retrievable through the RAG system, which tools the agent had the ability to invoke, which downstream systems trusted the agent's output without independent verification, and which other agents received and acted on output from the compromised session. Each of those four questions can expand the scope of the incident well past the single agent where it was first detected.

The complete prompt chain is the primary forensic artifact in this kind of investigation. Without it, a responder has no way to distinguish an agent that was manipulated by an attacker from an agent that made an autonomous decision the operator simply disagrees with, and that distinction determines everything about how the incident gets classified, reported, and remediated afterward.

Provenance tracking at the level of individual tool-call arguments, not just at the level of the tool call itself, is the technical capability that makes this kind of forensics possible. The leading research proposal in this space, Provenance-Aware Capability Contracts, assigns semantic roles to tool arguments and tracks the provenance of each value across every replanning step an agent takes. So you get the audit trail a forensic investigation needs afterward as a byproduct of the mechanism built to enforce trust contracts at runtime. Google DeepMind's CaMeL architecture takes a related approach, tracking the provenance of every value through a custom interpreter so that data arriving from an untrusted source cannot flow into a privileged tool call. The policy check that blocks that flow is itself a logged event, one that reconstructs precisely where a given argument originated and precisely which decision point let it through or stopped it.

Tool-call-level and parameter-level tracing, recording the tool name, its arguments, the source of each argument's value, the tool's output, any errors raised, and the downstream action taken as a result, is the minimum forensic infrastructure an agent deployment needs in place before an incident ever occurs. Investigation conducted after the fact cannot recover what was never recorded in the first place, and an agent's semantic chain, once lost, cannot be reconstructed from the event log alone.