Don't make the model rank it
On this page
Here are two problems found in the same repository, on the same run, by two different security tools. This is what each tool sent back, with the key itself and the committer's name removed.
{
"ruleId": "private-key",
"message": { "text": "private-key has detected secret for file app/config.example.pem at commit a103420…" },
"locations": [{ "physicalLocation": {
"artifactLocation": { "uri": "app/config.example.pem" },
"region": { "startLine": 1, "endLine": 8 }
}}]
}{
"ruleId": "yaml.github-actions.security.github-actions-mutable-action-tag.github-actions-mutable-action-tag",
"message": { "text": "GitHub Actions step uses a mutable tag or branch reference. Tags and branch names can be silently repointed…" },
"locations": [{ "physicalLocation": {
"artifactLocation": { "uri": ".github/workflows/draugr.yml", "uriBaseId": "%SRCROOT%" },
"region": { "startLine": 35, "endLine": 35 }
}}]
}The first is a private key committed to the repository. Somebody who copies it can sign things as you. The second is a build step that says "use version 4" instead of naming an exact version. That one is untidy and worth fixing whenever somebody gets to it.
You fix the first today and the second eventually. Now find the part of either message that tells you which is which.
So both problems arrive looking alike, and the only thing separating them is a sentence of English. Whoever reads that sentence is choosing what gets fixed first. Hand the file to a model and you have chosen the model.
That is the most expensive way there is to sort a list, and the only one that sorts it differently each time you ask.
Finding things is the easy part
The obvious move is to point a model at your code and ask what is wrong with it. A team of researchers did exactly that, carefully, and published the result in a paper called Argus. Their tool read a large Java project, 841,000 lines, and reported back.
The traditional code-analysis tool they compared it against found nothing real on any of the seven projects either, so this is not a story about AI being worse. Both approaches failed at the same step, and it was not the step where things get found.
A list of places where something might be wrong is not a list of things to do.
Producing the list is cheap and solved, whether a scanner does it in a few seconds of computing or a model does it for the price of tokens. Deciding which entries matter is the part nobody has automated, and the reason is the same for both.
What decides is not in your code
Three things make a problem urgent: how bad the bug is, who can reach the thing it sits in, and what the business loses when that thing goes down.
Only the first is in the code. Whether a service faces the public internet, and whether orders stop when it goes down, are facts about your architecture and your company. Nothing in the file the model is reading contains them, and a better model will not change that.
So Draugr asks you once, in a small file beside your code, where a reviewer sees it in the pull request:
components:
- name: api
exposure: public
criticality: importantDraugr combines those two lines with how severe each bug is, using two published tables, and that produces the ranking. Nothing guesses at any point in it. The same declarations and the same published advisories give you the same answer today, next month, and in next year's audit.
A model will not give you the same answer twice
Snyk built a test: ten projects, each with a known list of the bugs actually in it. They ran the same scan 300 times and compared every run against that list.
A serious problem that appears in three runs out of five is one your build lets through 40% of the time.
The quieter failure is what this does to marking something as accepted. An ordinary scanner's false alarm is the same false alarm on every run, so somebody reviews it once, marks it as accepted, and never sees it again. A model words its false alarms differently each time, so there is nothing steady to mark. Marking things as accepted is how every team survives a backlog it inherited, and it stops working.
Has to come out the same every time
What was found, how severe it is, the order, and whether the build fails. These decide what ships, who accepted what, and what an auditor is shown next year.
Can vary freely
The explanation, the wording, a draft fix, the argument in an exception. A second opinion here is useful, and nothing further down breaks because the phrasing changed.
No amount of prompting makes a model repeatable. It works by picking from a range of plausible answers, and picking better answers does not turn into picking the same one every time.
Looking at everything is expensive
Here is one scan of draugr-demo, our deliberately vulnerable public sandbox, on 2026-10-05. Nine tools, 10.82 seconds. P1 is the top of the ranking and P4 the bottom.
DRAUGR FAIL draugr-demo 1.0 10.824s
221 P1 650 P2 232 P3 18 P4
CONTROLS
iac FAIL 4 P1 5 P2 15 P3 16 P4
images FAIL 191 P1 267 P2 37 P3
licenses FAIL 353 P2 178 P3
provenance pass no findings
sast FAIL 7 P1 8 P2 2 P4
sca FAIL 18 P1 17 P2 2 P3
secrets FAIL 1 P1
MEASURED AGAINST
provenance cosign · coverage: 1 of 2 images checked, 1 unsigned · scope: 1 signer ·
pinning: none, all by tag
sca govulncheck v1.8.0;db@2026-10-01 20:24:15 · database: local copy, fetched
2026-10-04
COMPONENTS
api FAIL 217 P1 646 P2 228 P3
storefront FAIL 4 P1 4 P2 1 P3
platform pass 3 P3 18 P4
SIGNALS
KEV 2 findings raised
EPSS 3 findings raised
reachability govulncheck · 2 reachable, 2 unreachable
ACCEPTED
config.exclude 1 finding suppressed
VEX 1 finding excusedThat run found 1,121 problems. Here is what each version of them costs to hand to a model.
What you could give the model
The bars are to scale. The summary is about one hundredth of the full list, and it answers the question a model asks most often: am I blocked, and how badly?
A model working through a task does it in steps, and every step re-sends everything that came before it. Give it a large file at step one and you pay for that file again at step two, and at step three, and onward. Caching makes each repeat cheaper without making it stop happening.
One team instrumented a run over a single 800-line pull request and was billed for 156 million tokens, about $41, of which the model's own writing was 0.19%. That is one team's measurement rather than a controlled study.
What a price does to behavior matters more than what it does to the invoice. At twenty dollars a run, nobody runs it twice, and a result nobody re-ran is a result nobody checked. A scan you can repeat for a few seconds of computing is a scan whose second answer you can compare against its first. Draugr remembers the answer for a given commit, so asking again at that commit returns what it worked out the first time.
Give the model the other job
Draugr's MCP server hands a model the 25,879-byte summary first, then more of the ranked list when it asks, then the detail of one problem. It never sends the catalog of checks. Everything that comes back is structured: the rank, what produced it, which tool said so, the file and the line, and the reason recorded against anything already accepted.
With that underneath it, a model spends its budget on the work that needs judgment.
| Ask it to | Because |
|---|---|
| Explain this to the team that owns the service | Nobody there reads security scores, and explaining is what models are good at |
| Draft the upgrade and the tests around it | A version bump that might change behavior is exactly the kind of writing to hand over |
| Write the exception, with a reason somebody can argue with | The reason is what an auditor reads. The model drafts it and a person signs it |
| Answer who accepted this, and when | The record already exists, so the model reads it instead of reconstructing it |
None of those decide whether the build fails. That was decided by two published tables before the model was called, and it will be decided the same way tomorrow.
The same model, handed two different things
| What reaches it | An ordinary scanner | Draugr |
|---|---|---|
| Shape | one format per tool, three ways of writing a file path | one shape, whichever tool found it |
| How bad it is | missing, or two scores that disagree | on every problem, one scale, decided once |
| Order | the model's, worked out afresh each run | published tables, nothing guessed |
| Your business context | not in the repository, not recoverable from it | declared beside the code |
| Something already accepted | the wording moves, so it comes back | stays accepted, with a reason and the commit that wrote it |
| What it has to read | everything | the answer |
Building that structure is not free. The Argus researchers spent eight times the tokens getting their list from 13,962 entries down to 19. Draugr pays that cost once per commit, in 10.82 seconds, and every model that asks afterwards reads the result instead of recomputing it.
Try it on your own code
draugr scan .You do not need to set anything up first. It works out which checks apply from what is in the
directory, runs them, and gives you one ranked answer. Then add exposure and criticality to the
components that have them, and watch which problems move.
If you want to see the difference for yourself, run those scanners directly and read what they hand back before Draugr touches it.
(Sandbox figures from a scan of draugr-demo on 2026-09-19, Draugr 0.127.0. Advisories move and rule sets grow, so re-run it rather than trusting the numbers. The 13,962 figure is from Liang, Xie et al., Argus: Reorchestrating Static Analysis via a Multi-Agent Ensemble. The repeatability figures are from Snyk, VulnBench JS 1.0.)