Tracking security findings as issues

In plain terms#

A team plans its work in an issue tracker such as GitHub Issues, GitLab issues or Azure Boards. A security scan writes its results to a CI log or a dashboard, and work that is not in the tracker does not get planned, assigned or closed.

Draugr, an open-source CLI that runs security scanners and gates a pipeline on what they find, puts the scan in the tracker. After each scan of the default branch, Draugr opens an issue when the gate fails, rewrites it when the findings change, and closes it when the gate passes. With children turned on, each fix also gets an issue of its own under that one, so it can be assigned and closed like any other piece of work.

Why it matters#

Filing findings by hand usually goes one of two ways, and both leave a tracker that no longer describes the code:

A tracker earns trust when every issue in it describes the latest scan, which takes something that rewrites the issue on every run.

What an issue should track#

An issue tracks the gate, the rule a team has agreed to hold. A finding is too small a unit, since one base-image update can clear hundreds, and a scan is too frequent, since a nightly schedule would file a ticket a night.

Draugr keeps one issue open for each part of the run that fails, and every run on the default branch brings it in line:

An unchanged run writes nothing, so a nightly scan that found nothing new notifies nobody. A pull request changes nothing either, because the issue describes the default branch and an unmerged branch has not changed it.

The body groups the findings by the action that clears them, highest priority first, so the issue reads as a list of things to do. The demo's 561 findings come to 46 actions. The top of the platform team's issue, with each action's findings folded away:

561 findings fail the gate · gate P1, P2 for licenses · 2 accepted

P1  Update python:3.8-slim · images · 487 findings
P1  Upgrade jquery 1.8.3 · sca · 7 findings
P1  Upgrade Jinja2 2.10 · sca · 6 findings
P1  Upgrade requests 2.19.1 · sca · 5 findings

The issue carries a priority label, draugr:priority:P1, naming the highest band it lists, so a board or a saved search can sort by it. Priority differs from severity by folding in the exposure and criticality each component declares, so the same flaw ranks higher on an internet-facing service than in a batch job nothing reaches.

An issue per fix#

An issue holding 46 actions tracks the gate and is no use for assigning work, because nobody can be assigned one line of an issue body. children: actions makes each issue a parent with a child for every action:

config:
  publishers:
    - kind: github-issue
      children: actions     # none (default) | actions | controls
      maxChildren: 10       # 100 when unset

A child's title leads with its priority, so the forge's list of children reads in the order to fix them:

P1 · Update python:3.8-slim
P1 · Upgrade jquery 1.8.3
P1 · Upgrade Jinja2 2.10
P1 · Upgrade requests 2.19.1

Its body is that action's findings table. A child closes, with a comment, when the run no longer reports its action, and when the gate passes every child closes before its parent. children: controls gives one child per control instead, holding that control's actions, which suits a team where dependency upgrades and infrastructure fixes go to different people.

maxChildren is the most children open under one parent at once. Draugr creates them highest priority first, and the parent lists the actions that have none yet, opening with a line such as:

10 of 46 actions have an item. The other 36 are listed below and get an item as open items close, since maxChildren is 10.

A first run on a repository with a long backlog files the ten most urgent fixes rather than all of them, and each child that closes makes room for the next action in priority order.

Each forge links a child to its parent its own way:

Different teams, different issues#

An issue covering a whole organization has no owner. select gives an entry the part of the run it covers, matched on the labels each component declares, and split opens one issue per control or per component within it. Each issue closes when its own part passes, while the run may still fail on another team's:

config:
  publishers:
    - kind: github-issue    # the payments team's components
      select: { labels: { team: payments } }
      item: { assignees: [payments-lead] }
    - kind: github-issue    # license findings, for legal review
      select: { controls: [licenses] }
      label: draugr-licenses
    - kind: github-issue    # one issue per component
      split: component
      minPriority: P2

The title names the part an issue covers, then the project, as in these titles from the demo:

team=platform fails the Draugr gate · draugr-demo
team=web fails the Draugr gate · draugr-demo
sca fails the Draugr gate · data-class=pii · draugr-demo

When an entry sets select, draugr doctor names any component that no issue entry covers, so a new service does not go untracked because nobody added it to a team's selection:

UNTRACKED
  batch
  No issue publisher selects this component.

Common pitfalls#

Closing the issue by hand. Closing an issue does not fix what it lists. Draugr reads only open issues, so one closed by hand while its part still fails stays closed, and the next failing run opens another. For a finding the team has decided not to fix, record an acceptance with a reason and an expiry date, and the issue stops listing it.

Writing in the body. Each run that finds a change rewrites the body, and the fact labels follow the findings, so a priority label removed by hand comes back. Discussion belongs in comments, which Draugr leaves alone.

Publishing findings where anyone can read them. An issue on a public GitHub repository is public, and its body names each flaw and the file it is in. gitlab-issue creates each issue confidential, visible to project members only, unless item.confidential: false is set. On a public repository, select can limit the issues to controls such as sca, whose findings anyone can already work out from the repository's manifests and the public advisories.

A token that cannot write issues. GitHub's GITHUB_TOKEN starts read-only in new organizations, and GitLab's CI_JOB_TOKEN cannot write issues at all. The run fails and names the missing permission rather than passing with no issue written.

How Draugr fits#

Draugr writes to each tracker through a publisher, an entry under config.publishers in the descriptor that describes the project:

All three read select, split, minPriority, children, maxChildren, label, labelBy and branches. item takes what each tracker adds, such as assignees and a milestone on GitHub, confidential on GitLab, and the area and iteration paths on Azure Boards. Every field is in the descriptor reference.

The publishers run at the end of draugr scan, so wherever the scan already runs in CI, the entry and the token's permission are all it takes. The GitHub Action, the GitLab template and the Azure Pipelines guide each set up the scan itself.

An issue publisher sits beside the rest of what a scan writes. The pull-request comment answers for the change under review, and the issue answers for the default branch, so between them a finding has a place to be seen before it merges and a place to be worked after.

Keep learning#

• Vulnerability prioritization covers the band each issue and child is labeled and titled with.
• Risk acceptance and exceptions covers how a finding leaves an issue without being fixed.
• Gating a pull request on what it introduced covers the check that runs before a change reaches the default branch.
• Shift left, and what it cost covers where each check belongs in a pipeline.