Agents Lack Judgment: The Rules Problem in Agentic Penetration Testing

On October 19, Kevin Mandia went on CNBC and said the quiet part out loud.

Asked about liability and agents running loose, he argued that agents are not human and lack judgment. So they need definitive rules for what they can and cannot do. Training and classifiers built with domain expertise. Monitoring, in case one goes rogue. His phrase for the whole apparatus was "cage the beast."

I agree with almost all of that. I want to talk about the part the argument skips.

First, credit the concession

Three days earlier, on Fox Business, the same man said, "We can't have humans in the loop on security," because the fight is just too fast now.

Hold both statements at once, because together they are more honest than either one alone. Speed is real. An agent works a surface at a pace no human team sustains, and any defense that waits for a person to approve every step will lose ground. I run agents for exactly that reason.

Judgment is also real, and it does not come from the model. When the most prominent voice in this category goes on national television and says his own agents lack it, believe him. That is a practitioner describing his tooling, and it lines up with everything I have seen running mine.

So the disagreement here is narrow. He says the answer to missing judgment is definitive rules. I say rules are judgment, written down in advance, by somebody. The argument just moves upstream.

Rules encode judgment. They do not replace it.

A rule is a decision somebody made before the engagement started, about a situation they predicted. Consent and approved scope. These assets, not those. Stop when you reach production data. Escalate before anything irreversible.

Every one of those lines took judgment to draw. Somebody had to know the target class, the failure modes, and the cost of being wrong. Somebody had to decide what "too far" means in an environment they had never tested, against an adversary they had not met yet.

That is the work. Writing the cage is adversary reasoning done in advance, under calm conditions, instead of in the middle of a run. A vendor who hands you a cage built by people who have never run an operation has handed you their best guess, formatted as a control.

And the cage only covers the situations its builders imagined. The finding that matters in a real engagement is usually the one nobody predicted: the chain that crosses three systems, the trust relationship that exists on paper nowhere, the workflow a business team built around the official process. There is no definitive rule for the novel case. There is a decision, made by someone, in the moment. Call that whatever you want. It is judgment.

Mandia made a version of this point himself, in the same interview. He drew a line between "book smart" and "street smart" - lab researchers chasing the big prize with security as a secondary concern, against operators who have been through incidents and price the adversary correctly. I will take that distinction and apply it one level down. Rules written by the book-smart side of the house will be complete, consistent, and wrong about the three things that matter in your environment. Rules written from incident scar tissue look different. Ask who wrote yours.

A rule the agent can talk its way out of is not a rule

There is a second question underneath the first one: where does the rule live?

If the definitive rules sit in a system prompt, they are instructions the model can weigh against other instructions. Models get talked out of their instructions. That is a documented attack class in my line of work, not a hypothetical. Prompt injection, tool poisoning, a poisoned document in the data path: the whole offensive playbook against agent systems is, at bottom, persuading the agent that the rule does not apply here.

A rule enforced in code cannot be persuaded. An allowlist checked before every tool call. A scope boundary the agent cannot see past. A privileged action routed through a separate approval path the agent does not control. Depth gating and scan pacing as engineering controls, not as suggestions in a prompt. That is the difference between a guardrail and a hope.

This has been my position on agent systems for a while now, and it applies to my own shop first. If a guardrail is not executable, it is not a guardrail. So when a vendor tells you their agents operate under definitive rules, the follow-up is mechanical: show me where the rule is enforced. In the prompt, or in the code? One of those answers is a control. The other is marketing copy with a compliance accent.

Monitoring has the same problem

The cage doctrine ends with monitoring: watch the agent the way you would watch an employee who could go rogue.

Fine. Who watches, and what are they looking for?

An agent going rogue rarely looks dramatic. It looks like a thousand reasonable steps that add up to the wrong engagement: scope creep by inches, a target model drifting away from the actual environment, effort poured into the findings the tooling is good at finding instead of the path an adversary would actually take. A dashboard will show you green the whole way down.

Catching that takes a person who knows what right looks like in that environment. Monitoring is judgment applied after the fact instead of before it. You have not removed the human from the loop. You have moved the human to the two places judgment is hardest to fake: writing the rules before the run, and reading the run honestly after it.

It is also, described plainly, a human in the loop.

What to ask before you buy

If the category leader is telling you agents lack judgment, diligence gets simpler. You are no longer buying judgment from the machine. You are buying the quality of the judgment wrapped around it. Five questions price that:

Who wrote your rules of engagement, and what operations have they personally run? The rules are the product. Staff them accordingly.
Where is each rule enforced - in the prompt, or in code the agent cannot argue with?
What happens when the agent reaches a situation no rule covers? "It stops and a human decides" is an answer. Anything smoother is a warning.
Who monitors the run, and can they show you a run they stopped and why?
When did a finding last change the ruleset itself? A cage that never gets rebuilt is a cage built for last year's adversary.

The bottom line

Mandia is right that agents lack judgment. He is right that they need definitive rules, real scope discipline, and monitoring. Where I part company is the implication that rules settle the question.

Rules keep an agent inside the lines. Judgment decides where the lines go, and it decides again every time the environment does something the lines never anticipated. Buy the cage. Then ask hard questions about who built it, what it is made of, and who is standing next to it while the agent runs.

The beast was never the hard part. The cage builder is.

References

CNBC, Squawk Box, "Armadin CEO Kevin Mandia on defending against AI cyber threats, future of AI agents," aired October 19, 2026. https://www.cnbc.com/video/2026/10/19/armadin-ceo-kevin-mandia-on-defending-against-ai-cyber-threats-future-of-ai-agents.html
Fox Business, The Claman Countdown, "Armadin CEO warns of China-linked BeyondTrust THREAT," October 16, 2026. https://www.youtube.com/watch?v=ArcgXLxrmMg