Gate PRs on new findings

draugr diff compares two scans and classifies every finding as new, fixed, accepted, reopened or unchanged, the security delta of a change, typically a PR's head vs its base branch. This lets you gate a PR only on the findings it introduces, not the pre-existing backlog, so the gate stays adoptable where a whole-backlog gate would block every PR.

How it works#

Draugr stores nothing. There is no baseline kept on a server, no history of previous runs, no "last known state of main". draugr diff takes two SARIF files that you hand it and compares them:

draugr diff base/results.sarif head/results.sarif

So "the result from main" is a file you produced by scanning main. In the same pipeline run, or stored as an artifact by the last build of main. Nothing is fetched.

That is deliberate. A CLI running in someone's pipeline should not be a service with memory of previous runs, because then the answer depends on state you cannot see and cannot reproduce. Two files in, one answer out, the same answer forever.

What counts as "the same finding"#

Findings are matched on tool + rule + file + message, deliberately not on the line number or the severity. Code moves, and a finding that slid down twelve lines is not a fix plus a new problem. A CVE that gets re-scored is still the same CVE.

Whatever is in head and not in base is new; in base and not in head is fixed; in both is unchanged.

Two more, for the findings somebody decided about rather than changed:

• accepted, suppressed in head and not in base. Somebody added an exclusion, or a finding arrived that an existing rule already covers. Accepting a risk is not fixing it, and this is the line most worth a reviewer's attention: nothing was removed, somebody chose to live with it.
• reopened, suppressed in base and counting again in head. An exclusion was removed, or it reached its expires date. Nobody introduced this finding; a decision about it lapsed, and reporting it as new would lose the part that needs acting on.

Both are printed only when they are not zero, so a diff with neither reads exactly as it always has.

Draugr diff · 0 new, 0 fixed, 1 accepted, 0 unchanged

Accepted (1) · still present, somebody decided to live with them:
  ~  high  CVE-2024-11111  requirements.txt:3

Where the base comes from#

Three ways, in increasing order of effort:

The middle one works on any CI system and is the one to start with. The artifact approach is faster, at the cost of a base that describes whatever commit last ran rather than the actual merge base.

In CI: let the action do it#

On GitHub, you don't wire this up by hand. The first-party action's default mode: auto runs a diff on pull_request events. It scans the base and head for you and posts one sticky new/fixed comment, and a full scan on push. One workflow, one Saga:

on: [push, pull_request]
permissions:
  contents: read
  security-events: write   # push: code scanning
  pull-requests: write     # PR: the diff comment
jobs:
  draugr:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }        # diff needs the base commit
      - uses: draugr-dev/draugr@v0
        with:
          saga: draugr.saga.yaml
          tools: true
          fail-on-new: high             # gate only on findings this PR introduces

See the GitHub Action guide for all inputs and modes. The rest of this page covers running draugr diff directly, for other CI systems, or to understand what the action does under the hood.

Produce the two SARIF files#

diff consumes the results.sarif files that draugr scan -o writes (SARIF is the complete, structured result set). A typical setup scans main on push and stores results.sarif as an artifact, then scans the PR:

draugr scan draugr.saga.yaml --no-gate -o base/    # on the base branch
draugr scan draugr.saga.yaml --no-gate -o head/    # on the PR head

--no-gate on both. These two scans exist to produce reports; the diff is the gate. Without it a FAIL verdict on the base, which any repository with a backlog will produce, exits non-zero and takes the whole step with it under set -e. It suppresses the verdict's exit code only: a scan that could not run still fails, so a missing report never reaches the diff disguised as "no new findings".

For a complete pipeline, see Azure Pipelines; on GitHub the action's mode: auto does all of this for you.

Each scan clones the repository before reading it, so a results.sarif always describes a committed revision, which is what makes the two comparable, and what a reader needs in order to reproduce either side. It also means the pair above only differs if the two scans ran against different commits: iterating locally with scan → edit → scan compares HEAD with itself and reports no change. Commit between the two, or set revision on the repository to name each revision explicitly. See URLs and paths.

Diff and gate#

draugr diff base/results.sarif head/results.sarif                     # console delta
draugr diff base/results.sarif head/results.sarif --format markdown   # MR comment
draugr diff base/results.sarif head/results.sarif --fail-on-new-priority P1
draugr diff base/results.sarif head/results.sarif --publish           # sticky PR comment (in CI)
draugr diff base/results.sarif head/results.sarif --format sarif      # only the new findings, for code scanning
draugr diff base/results.sarif head/results.sarif --format sarif --min-priority P1

--format sarif writes the new findings and only those, which is what a pull request's review comments should carry: an upload of the whole repository annotates a reviewer with hundreds of findings the branch did not cause. Fixed and unchanged are deliberately absent. A fixed finding is no longer there to annotate, and an unchanged one is the pre-existing noise this removes. The GitHub Action does this for you; see code-scanning.

--min-priority narrows the new findings it reports, in any format, leaving fixed and unchanged counts alone. Narrow the diff rather than the scans it came from: a diff computed on filtered inputs reads every finding the filter removed as fixed.

--fail-on-new / --fail-on-new-priority fail the command (non-zero exit) only for new findings at or above the given severity / priority. Findings are matched on (tool, rule, file, message), deliberately ignoring the line number (which drifts as code moves) and the severity level (a re-scored finding is still the same issue), so genuinely-carried-over findings aren't reported as fixed + new.

Post the delta as a PR comment#

--publish posts the diff as a sticky pull-request comment, updated in place on each push, and no-ops off a pull request:

draugr diff base/results.sarif head/results.sarif --publish

It picks the publisher from the CI system it is running on, github-pr-comment under GitHub Actions with $GITHUB_TOKEN, azure-pr-comment under Azure Pipelines with $SYSTEM_ACCESSTOKEN, gitlab-mr-comment under GitLab CI with $GITLAB_TOKEN. Azure needs that variable mapped into the step; see Azure Pipelines. GitLab needs a token with api scope, because the CI_JOB_TOKEN in every job cannot post notes; see reports & publishers.

The diff keeps its own sticky comment, separate from the one a Saga's PR-comment publisher maintains. A pipeline can run both, the state of the branch, and what this pull request changed, and get two comments rather than one overwriting the other.

Severity in a diff#

A diff reports the same critical / high / medium / low bands the scan report uses, because it is read next to that report and the two have to agree.

Those bands are Draugr's own, normalized across every control so a dependency CVE, a leaked secret and an IaC misconfiguration can share one ordered list, a CVSS-style score decides the band when a scanner publishes one, and the SARIF level decides it when none is published. The error / warning / note values you will see inside a results.sarif file are SARIF's wire vocabulary, not a severity: SARIF has three of them, and they cannot express the difference between a 7.0 and a 9.8.

--fail-on-new takes a severity band (critical / high / medium / low), the same words the diff prints, and the same the scan gate takes. The SARIF levels error, warning and note are still accepted and mean high, medium and low.

Severity is still not priority. P1–P4 fold in the component's declared exposure and criticality, which is why a high on an internet-facing component outranks a critical on something nothing can reach. See prioritization.

See the CLI reference for every diff flag, and reports & publishers for both publishers.