What AI Pentest Agents Get Wrong

Every few months another launch video drops. An agent maps your attack surface, fires exploits, writes the fixes, re-tests, and reports back before lunch. The demos are genuinely impressive. I am not here to tell you the machines do nothing.

I am here to tell you they are wrong about what the job is.

First, what the machines actually give you

Credit where it is due, because the concession matters. An agent corps gives you three things no human team sustains: speed, scale, and continuous coverage.

Speed: recon across a surface that would take a human team a week, done before the scoping call ends. Scale: thousands of requests correlated into evidence without anyone getting tired or bored. Coverage: the thing runs on a schedule that does not care about headcount, holidays, or ticket queues.

I know this because I live it. I run a security program at a maturity that normally takes a multi-million-dollar org, as one person, with an AI agent corps under a deterministic orchestration layer. Every agent has a mission, a least-privilege tool scope, an autonomy tier (act, draft, or escalate), an audit log, and a kill switch. The machines do the high-volume, noisy work that needs semantic judgment. The deterministic parts stay code. A human sits at every privileged action.

Imagine an army of personal NSA hackers working for your company. That is the promise, and the promise is real. It is just not the whole job.

Probing like a hacker is not thinking like one

Here is the category error, and it is the same one I keep finding in this industry: the vendors measure autonomy. How much of the engagement runs without a human touching it. That is the wrong unit of measure.

The right unit is judgment coverage. How much adversary reasoning is applied to your environment per engagement hour. Those are not the same number, and maximizing the first does not maximize the second.

A scanner, however agentic, enumerates. It walks the surface and reports what responds. An operator builds a theory of the target and then tries to break the theory. That is the falsification turn, and it is the whole game. Confirming that a vulnerability exists is cheap. Falsifying your own model of how the target behaves, finding the assumption the defenders never examined, chaining three low-severity findings the scanner filed separately into the one path that actually matters: that is adversary reasoning, and it does not emerge from running more probes.

"Probes like a hacker" is a claim about behavior. Thinking like one is a claim about cognition. The launch videos demonstrate the first. They assume the second.

I found the limit by hitting it

This is not theory for me. I spent eleven days building a multi-agent workforce, got a three-hop delegation chain working, and then shelved the entire thing with a written post-mortem on exactly where autonomous delegation stops working.

The finding: each delegation hop loses intent. The top-level goal survives, but the context that made the goal sensible degrades with every handoff. By hop three the agents are executing competently against a plan that no longer quite matches the situation. Nobody notices because every individual step looks reasonable. The failure is in the composition, and composition is where the operator lives.

That is the shape of the risk in every "fully autonomous" pentest claim. Not that the agent does something dramatic and wrong. That it does a thousand reasonable things that add up to an engagement shaped like the vendor's training data instead of shaped like your adversary.

Ask where the human sits

There is a simple test, and I recommend buyers run it on every vendor in this category including me. Ask, in writing: where exactly does the human sit in your loop? Which decisions can the agent make alone, which does it draft for review, and which are gated to a person?

The serious firms answer crisply, because they have thought about it. The autonomy tier is act, draft, or escalate, and they can show you the gates. The firms selling replacement get vague, because vagueness is the product. Full autonomy in an engagement is a marketing claim, not a methodology.

I apply the same standard to my own shop. Agents where they earn it: high-volume, noisy work that needs semantic judgment. Code where it counts: deterministic collection, guardrail enforcement, anything irreversible. A person where it matters: scoping, chaining, verification, the privileged actions. If a vendor cannot draw you that map, keep shopping.

The multiply frame

So here is the honest version of the pitch, and it is the one I will keep making.

The autonomy vendors sell AI replacing the hacker. I sell AI multiplying the hacker. The army is real. I want the army. I run the army. But an army without a general is just expensive noise pointed at your infrastructure, and the general is the part that thinks like the adversary.

The machines give you the pace, the coverage, the scale no human team sustains. The operator gives you the theory of the target, the falsification turn, the chain nobody's scanner filed because no single finding looked like much. You need both.

The vendors are selling you the army without the general. Keep the army. Hire the general.

---