Skip to content

Detection Engineering Against AI Agents, What Is Changing

Detection engineering is not failing. It was specified for conditions that are increasingly becoming legacy.

Security leaders on how detection engineering is changing against agentic threat actors
Design by Maxim Berg / Unsplash
Published:

Detection engineering runs on a set of assumptions nobody bothers to write down, because until recently nobody had to. That attackers reuse their tooling often enough to fingerprint it. That normal behaviour holds still long enough to baseline against. That a security team gets minutes, sometimes hours, between the first odd event and the one that actually matters. And that somewhere behind the keyboard there is a person who gets tired, gets curious, and makes recognisably human mistakes.

Every one of those is now under pressure, and the pressure is uneven. Breakout time has collapsed to seconds, which strips out the response window the discipline assumes. Agents leave tells worth hunting today, though those tells are artefacts of immaturity rather than anything permanent. Payload reuse has stopped being necessary, which takes the economics out from under indicator-based rules. Baselines never settle in environments deploying agents weekly. And an agent's own authority turns out to be a more reliable signal than anything it does once it starts running.

Talk to the people who build detections for a living, or who build the agents doing the attacking, and they converge on the same diagnosis. Agentic attackers did not invent new techniques. They broke the operating conditions the whole discipline was specified around, and most detection programmes have not caught up to that distinction yet.

Security Teams Have Seconds, Not Minutes

Breakout time measures the gap between an attacker gaining access and moving laterally through an environment, and it is the closest thing the industry has to a shot clock. Jason Rivera, a former US Army intelligence officer who ran offensive cyber operations before moving to defensive readiness work at SimSpace, tracks that number as it collapses.

"I remember when I first joined CrowdStrike in 2018, the fastest breakout time was like eighteen minutes and fifty-one seconds," he shares. "Last year, the fastest breakout time was twenty-seven seconds. And so think about that, right? In a seven-year time span, it went from eighteen minutes and fifty-one seconds to twenty-seven seconds. And the very obvious question I'll throw to you in the audience, where does that breakout time go in the next five years? Right above zero. And so let's think about that for a second. What does the world look like when the breakout time is right above zero? There is no more alert triage. There is no more incident response. Or the way that we think of these things ceases to function in their current state."

That is not a rhetorical flourish, and it deserves to be treated as an engineering constraint rather than a warning. Every defensive function in a standard programme carries a built-in assumption of elapsed time, and the whole operating model degrades as that time approaches zero. So the first useful exercise is arithmetic rather than procurement. Map your detection and response chain end to end and put a real number on each hop, from event generation through enrichment and analyst triage to containment. Then set that total against twenty-seven seconds. Any control that only functions because a person reviews it inside that window needs automation behind it or a redesign, and knowing precisely which of your controls fall into that category is worth more than another sensor.

The compression runs upstream of the intrusion too, into the window between a vulnerability going public and being exploited at scale, which in cases documented by Sysdig's threat research team has narrowed to as little as four hours. Sysdig did not sponsor this content.

Offensive Eng #4: Cloud Security Crosses Into Machine Scale
Michael Clark and Crystal Morin on the four-hour exploitation window, an identity layer that is 97% machine, and the automated response almost nobody switches on.

Sysdig did not sponsor this issue

Stop Trying to Detect the Driver

The instinct, once teams accept the speed problem, is to build detections that identify AI involvement. Find the artefacts that betray a model, alert on those, and you have a handle on the new threat class. Sachin Jade, chief product officer at Cyware, argues that framing is a dead end before it starts.

"How to detect an AI attack, in my opinion, is the wrong question," he reasons. "The strategic move is to stop classifying only the driver and instead detect the behavior chain in a way that fires regardless of whether a human or an agent is at the keyboard, because within a year or two, a throttled agent that behaves like a patient human could be the norm. The adversarial tooling and actor can change but their objective still remains the same. It indicates behavior, and behavior survives obfuscation."

Which is the part worth sitting with, because it changes what a durable detection looks like. Everything currently marking an agent out is an artefact of immaturity rather than a permanent property. Operators are moving fast and have no reason yet to slow their tooling down, so the tells are loud. The moment a reason appears, the tells go quiet, and any logic built on them stops firing while the attack underneath continues exactly as before. So write detections against objectives rather than operators. Model what an adversary is trying to achieve as a behavioural chain, from credential access through persistence and lateral movement to exfiltration, and make every stage fire on its own terms. When you do build something agent-specific, tag it as a temporary signal and put a review date on it, the way you would with any indicator you expect to decay.

Agent Tells Are Real, but They Are Temporary

None of which makes today's tells useless. They work now, and teams should use them while they last, provided nobody mistakes them for foundations.

Michael Clark, who leads threat research at Sysdig, finds them in small operational details rather than in novel behaviours. "It doesn't look all that different, honestly, than typical attacks, because the same things have to happen," he highlights. "However, there are keys that get given away that it's an agent. An agent needs to be able to read machine-readable output. So you can see how the attacker's doing something, they try to format it in a certain way so their agent will be able to easily read it, versus if it was a person and just their scripts, they don't necessarily care that much. We see custom scripts that have the target hostname built in and data about the host built into the script. Usually you wouldn't see that if it's a human. It would be a generic script being run, not one tailor-made with the values only for that host."

Those are cheap detections to build and worth building this quarter, because they cost little and buy time. Hunt for host-specific values compiled into scripts where a generic tool would do, and for output formatted more carefully than any human operator would bother with. AI-generated code also tends toward the verbose and heavily commented, which leaves strings a hunter can key on even when the behaviour underneath looks unremarkable. Just record them in your detection inventory as short-lived, because the moment operators start throttling their agents to blend in, every one of these goes dark at once.

Tempo and reach tell a similar story from a different angle. Ankita Gupta, co-founder and chief executive at Akto, works on the offensive side of that question during red-team engagements.

"Human attackers have think-time baked into everything they do. They pause to read a response, second-guess themselves, get distracted, try something dumb, backtrack," she explains. "Agent-driven attacks don't have any of that. What you see instead is a session where the recon, the hypothesis, and the exploit attempt all happen back-to-back with almost no gap between them, and that pace never changes, whether it's step three or step three hundred. A person gets tired or curious eventually. An agent just keeps going at the exact same speed. The second thing is breadth. A person tends to go deep on one lead because their time is limited and expensive. An agent doesn't have that problem, it can cheaply try dozens of versions of the same idea at once."

But the difference that should reshape how teams scope detection is imaginative rather than mechanical, and it showed up in one of Gupta's engagements against a customer's AI-powered support tool.

"Instead of going after the API directly, the agent noticed the support tool could touch both a knowledge base and a ticketing system. So it planted a crafted message inside a support ticket, which got pulled into the knowledge base. Then, in a totally different session later on, it referenced that planted content to change how the support agent behaved," she recalls. "No person would've gone looking for that. Nothing about how people normally think about attacking a support tool says hide something in a ticket now so you can use it later. The agent found it because it saw the two systems as one connected thing, not two separate products."

So draw the map that example implies, and draw it before an agent does it for you. Chart which of your systems can write into which other systems, paying particular attention to anywhere content flows automatically from a low-trust surface into something an agent later treats as authoritative. Ticket queues feeding knowledge bases. User-submitted content reaching a retrieval index. Logs that a downstream tool parses as instruction. None of those crossings appear on an architecture diagram as a trust boundary, and all of them are exactly where an agent will probe, because it carries none of a human's assumptions about where one product ends and another begins.

Indicators Lost When Reuse Became Optional

That mapping exercise only pays off if the detections sitting on top of it are built to survive, which brings the argument to why the older model is losing ground. And the reason is not the one usually given.

Indicator-based detection was never really about signatures. It was about economics. Building new tooling cost time and skill, so attackers reused what already worked, and that reuse is what made fingerprinting viable in the first place. Generation is now close to free, and the premise collapses with it.

Gupta puts the collapse in terms of what teams stop being able to rely on. "An agent can generate a brand new payload, a new way of encoding it, a new sequence of steps, for every single target, basically for free," she points out. "There's nothing to fingerprint because nothing repeats. By the time you've flagged one version and added it to a blocklist, the attacker's already moved on to something that looks completely different but does the same thing. The shift I've seen in teams doing this well is they've stopped asking have we seen this exact thing before and started asking does this look like how something behaves."

Which sets up a concrete audit worth running before the next planning cycle. Work out what proportion of your alerting still depends on artefacts an adversary can regenerate at no cost. If most of your rules fire on hashes, addresses, and payload signatures, you know where the next quarter of engineering effort belongs. Indicators stay useful for cheap retrospective hunting and for blocking known infrastructure, so keep them, keep them inexpensive, and stop asking them to carry primary detection weight.

Crystal Morin, a senior cybersecurity strategist at Sysdig who came into the field from Air Force intelligence, describes what replaces them from inside a detection programme. "A really important part of being able to use automated response actions is having high fidelity detections," she notes. "So moving from IOC based detections to behavioral detections. So building your detections based off of several cases of what an attacker is doing, bringing in the identity based behaviors plus the scripts, plus several other things to build high fidelity detections."

Jade frames the same move as a question of inventory placement rather than replacement, arguing indicators are demoted rather than dead and belong now in the enrichment and context layer instead of the primary detection layer. And there is a practical benefit to that placement beyond tidiness. Much of the false-positive load in a behavioural programme comes from third-party intelligence about other organisations' incidents, which carries no causal relevance to your environment at all. Context is what turns a raw behavioural signal into something you would let a machine act on, so treat enrichment as a fidelity investment rather than a reporting nicety.


Baselines Never Get to Settle

Behavioural detection inherits a problem of its own, though, and it is more awkward than the one it solves.

Baselining assumes the population being modelled changes slowly enough to learn. In an environment deploying agents continuously it does not, and a model needing thirty days of stable behaviour to establish normal never catches up to a population turning over every week. Teams end up treating their own deployment velocity as anomalous, which trains analysts to dismiss the exact signal class the system exists to surface.

Mahesh Kumar Goyal, a senior data and AI engineer at Google, locates the failure in tooling built for a different subject entirely. "EDR tools are built to detect anomalies in the systems, or your web traffic, or in human behaviors," he observes. "But an agent that runs code perfectly maybe a hundred thousand times can make a mistake on the next occurrence. That's where identifying a specific anomaly in the agent is very tricky compared to identifying the anomaly in traditional systems. And all the systems like IAM, RBAC, DLP assume there's a human or a static service behind all of these operations. But with agents, these are dynamic."

He has also set out how memory and accumulated context inside an agent make the origin of a problem genuinely difficult to locate, because whatever went wrong may have entered the system long before it produced anything visible. And a second complication compounds all of it. Your own sanctioned agents enumerate systems, retrieve credentials from a vault, execute code they generated moments earlier, and touch a dozen services inside a minute. Separating that from an adversarial agent doing the same things for different reasons is close to impossible on behaviour alone.

So stop trying to solve it with statistics. Keep an inventory of every agent running in your environment alongside its declared capabilities, because the platform teams deploying agents usually hold a precise, versioned capability manifest that detection engineers have never thought to ask for. Baseline per agent identity rather than across the whole population, so a new deployment does not poison the model for everything else. And treat the identity enrolment event itself as high-value telemetry, since every sanctioned agent receives its identity at a specific auditable moment while an adversarial one has to acquire it some other way.

Watch Authority, Not Intent

Enrolment telemetry points at something larger, which is that the most reliable signal around an agent is not what it does but what it was permitted to do in the first place.

Kay Daskalakis, who advises enterprise leaders on identity attack paths at SpecterOps, argues the danger has almost nothing to do with whether an agent means well. "A human developer can notice something is wrong and refuse. An AI coding agent, at least now, executes within whatever scope it was granted," he cautions. "So it cannot contextualize a suspicious instruction. It cannot refuse. The risk is never that the agent is malicious. The risk is that a credential holder with no capacity to distinguish a legitimate task from an attacker-driven one is being put in the driver's seat. So the question is not whether the agent is human, it is what authority it can exercise."

Authority has one enormous advantage over behaviour as a detection surface, which is that you can know it in advance. Daskalakis extends the point to integrity itself, arguing you cannot establish it by examining the artefact. "You cannot verify the integrity of an artifact by checking the artifact itself. You verify it by checking everything that had the ability to influence its production." He calls that a graph problem rather than a hash problem, and the framing survives translation to any audience in the building, because few executives follow a discussion of token scopes while everyone understands that a printer should not be able to push code into production.

The work that follows from this is mapping every path to a privileged outcome rather than every privileged account. Which identities can trigger a build. Which workflows can assume a cloud role. Which service accounts can publish an artefact or rotate a secret. None of those appear on a standard access review, and every one of them reaches a privileged result indirectly. Once the map exists, control decisions stop being guesswork, and the first surprise is rarely who holds admin. It is who can become admin without anyone approving it.

Control the Execution Boundary

The people building agentic systems arrive at a similar conclusion from the opposite direction, which is a useful corroboration for any defender wondering whether authority-first thinking generalises.

Albert Ziegler, head of AI at XBOW, orchestrates roughly five thousand agents inside a single autonomous security assessment, and he holds no illusions about what he is orchestrating. "Agents are little agents of chaos, who are quite often making wild assumptions, hallucinating things, and just being factually wrong," he acknowledges. "The agent orchestration system's job is not to be confused by that. So whenever an agent thinks it obtained some result, it needs to be validated before it goes into any kind of knowledge base. Whenever any agent tries to hand off to another agent, first there must be a check whether that is a sensible handoff. Many early agentic systems are very unconstrained, where any agent can write to some shared memory and kick off other agents. And that is incredibly dangerous and very quickly runs into these cascading system failures."

His governance model borrows from detective fiction and knocks out motive, method, and opportunity together, on the reasoning that leaving any one of the three intact eventually produces the failure you were guarding against. Give an agent goals it can satisfy without doing harm. Grant only the methods it genuinely needs for the task in front of it. Then remove the opportunity to misuse whatever remains by checking actions at the moment of execution rather than trusting the reasoning that produced them, which is how his own agent fleet operates at scale.

Offensive Engineering #1: Agents with Offensive Capability
Albert Ziegler on autonomous security agents, offensive architecture, and the governance gap

XBOW did not sponsor this issue

And the asymmetry underneath that is the most useful thing in this entire conversation for a defender. Whatever an agent decides happens inside a reasoning loop nobody can fully inspect, and no amount of prompt discipline changes that fact. But whatever it does has to cross a boundary that can be instrumented, logged, and blocked. Controls sitting on that boundary hold their value as models improve, while controls depending on a model behaving sensibly get weaker with every release. Put the engineering effort on the boundary, validate results before they enter shared state, and make handoffs between agents an explicit decision rather than a default permission.

Security Practitioners Still Know the Terrain

Two advantages survive all of this, and both are structural rather than technical, which means they do not erode as models improve.

The first is knowledge of your own environment. An attacking agent works from the outside in, inferring your architecture and your normal through trial and error, generating noise while it learns things your own team already knows for free. Speed makes that gap more visible rather than less, because a fast search leaves a broader trail than a patient one.

The second changes what winning even looks like, and Gupta makes the case for it directly. "You don't need to catch the attack that works. You just need to notice the looking," she underscores. "An attacking agent only needs one path to succeed. A defender doesn't need to wait for that. Good security practitioners just need to recognize the shape of the search itself. The volume, the parallel probing across systems that were never meant to be touched together, the pattern of something reaching across your environment faster and more consistently than a person ever would. That pattern shows up whether or not the attack ever lands."

Which is a genuinely different detection target, and a more achievable one. Build for the recon phase rather than the exploitation phase, alert on the shape of a search rather than on the success of an exploit, and you catch things before anything real gets touched. That reorientation costs nothing in tooling and quite a lot in habit, because it means accepting alerts on activity that never becomes an incident, which is exactly the sort of alert most programmes have spent a decade tuning away.

None of it arrives free, and Morin is blunt about the distance between building good detections and actually trusting them. Automated response sits configured and switched off across most organisations that have it, and confidence in detection fidelity is what stands between capability and use. The teams closing that gap are not waiting for better models. They are training detections on their own traffic rather than on someone else's incidents, staging response by blast radius so an early mistake costs almost nothing, and making sure an analyst can read a decision and understand in plain language why it fired.

Detection engineering is not failing. It was specified for conditions that are increasingly becoming legacy, and the work in front of every team now is writing those assumptions down and testing, one at a time, which of them still stand.

More in Agentic AI

See all

More from S Pattnaik

See all