Why we didn't build an AI pentester

Every few weeks now there's a story about a model finding a real vulnerability in real software, unaided. They're true, and they're going to keep coming. The reasonable question that follows is whether tools like this one have a future, and the answer we keep hearing from other vendors is to bolt a model onto the product and call it AI-powered security.

We went the other way. Draugr's core doesn't use inference at all, and won't. Here's the reasoning, because it's more interesting than the conclusion.

What AI commoditizes#

The thing models are getting good at is judgment on unfamiliar code. Reading a codebase nobody documented, noticing that two innocuous functions compose into something dangerous, chaining a foothold into an exploit. That's the pentester's craft, and it's expensive, scarce, and slow.

Making it cheap is a real change, and mostly a good one. But notice what it isn't. It isn't the job most security tooling does. Draugr doesn't discover novel vulnerability classes. It runs known checks against a known surface, every time you push, and tells you whether you can ship. Those are different products that happen to share a category name.

Four things a model can't give you#

Not "can't yet". These follow from what a language model is.

Determinism. Sampling is nondeterministic by design. That's a feature everywhere except the one place we care about, because a gate that returns different verdicts on identical input isn't a gate, it's an opinion. You can turn the temperature down. You cannot turn it into arithmetic.

Enumerable coverage. We can tell you exactly what Draugr checked, meaning these controls, these scanners, these versions, this commit. Nobody can tell you what a model looked at. "We believe it considered everything relevant" is not an answer that survives an audit, and it's certainly not one that survives a customer's security questionnaire.

Reproducible evidence. Control X, version Y, commit Z, verdict pass is an artifact with a chain of custody. You can re-run it and get the same thing. A narrative explaining why something seemed fine is not evidence, however well written.

Inner-loop economics. Draugr runs hundreds of times a day across a fleet, for fractions of a cent. Agentic assessment on every pull request is orders of magnitude more expensive. This gap is stable rather than temporary, because inference is getting cheaper while deterministic tooling is getting cheaper faster, and the ratio at the inner loop stays lopsided.

The equilibrium those four imply is deterministic tools at the inner loop, AI at the outer loop. We'd rather own the inner loop than pretend the distinction doesn't exist.

The risk worth planning for#

Here's the part the headlines miss, and the part we spent the most time on.

Our scanners were never the differentiated bit. Trivy, Semgrep and Gitleaks are excellent and free, and anyone can run them. What Draugr adds is the layer above, deciding which controls apply to which component, normalizing four tools' output into one schema, ranking findings by what they would cost you, and turning that into a verdict.

That layer is exactly the kind of glue an agent can improvise. "Run trivy and semgrep on this repo and tell me what matters" is a plausible prompt. If that's all Draugr is, an assistant routes around us in a year.

So the honest question was never can AI replace scanners, but what do we have that an improviser can't produce, and the answer turned out to be the same thing in every case. something written down and reviewed.

• The descriptor is committed. Ask an agent twice, you get two scopes. Ask Draugr twice, you get the scope your team agreed on.
• Priorities come from a component's declared exposure and criticality. Is this internet-facing? is not a question you can answer by reading source code. Somebody has to say so.
• The gate is a policy plus an exit code, not a recommendation.

Agents are strong at inference and weak at institutional memory. So we leaned into being the memory.

What we built instead#

If an assistant is going to answer security questions about your code, and it is whether or not anyone planned for that, the useful thing is for it to answer from your descriptor rather than from a scope it invented.

So Draugr speaks the Model Context Protocol. Your assistant asks Draugr which controls exist, what your Saga says, what the last scan found, and gets back the same prioritized answer your pipeline gets. Read-only by default, since scanning clones repositories and runs external tools, so it's offered only when you say so, and --scan=ask puts each one in front of you first.

That's the position, deterministic core and AI at the edges. Detection, severity, the gate and the evidence never involve a model, because they have to be reproducible. Explaining a finding to someone who's never seen one. That's judgment, and being approximately right is useful there.

The uncomfortable part#

Cheap autonomous exploitation is not good news, and we're not going to pretend our tool neutralizes it. Attackers get the same cost reduction defenders do, and probably use it more efficiently.

But the response to time-to-exploit collapsing isn't a smarter one-off assessment. It's closing the fundamentals continuously, on every commit, before the thing ships, which is a throughput problem, and throughput is what deterministic tooling is for.

Meanwhile AI is writing an enormous amount of code, held by teams with less security expertise per line than ever. More software, faster, from people who've never configured a scanner. Demand for boring, repeatable, hand-holding security tooling goes up in that world, not down.

We're betting on boring. Try it, which is Apache-2.0 and it runs entirely on your machine.