Split a Saga across files

A descriptor that has been running for a while is two things in one file: a structural account of the system, and a log of dated decisions about findings somebody accepted. They change at different times, for different reasons, and a developer adding a repository and a security owner accepting a CVE should not be the same review.

fragments: lets them live apart.

# draugr.saga.yaml
project: acme
release: { version: "3.1.0" }
fragments:
  - path: ".draugr/exclusions/*.saga-fragment.yaml"
components:
  - name: web
    repositories: [{ url: "." }]

# .draugr/exclusions/test-fixtures.saga-fragment.yaml
config:
  exclude:
    - paths: ["test/integration/repo_scan_test.go"]
      rules: ["private-key"]
      reason: >-
        The integration test writes a throwaway key so the secrets control has something real to
        find. The material is fake and was never valid anywhere.

The record of who accepted a finding is the commit that wrote its exclusion. A scan's report.json names each exclusion's file and line and the last commit to change that line, and draugr scan --evidence prints it under Decisions. The report schema says when that history is reliable, and Full history how to check out enough of it in CI.

What a fragment may contain#

A fragment adds scope, adds attributed suppressions, or contributes a setting that can only add findings. It cannot change policy.

That rule is what makes a one-line fragments: entry safe to review. Including a file cannot lower your gate or switch a control off; the worst it can do is add suppressions, and every one of those is attributed and counted in the report.

A setting is on that list only when contributing it can add findings and never remove them, when two files can both contribute without either losing anything, and when the report can say which file each contribution came from. signers qualifies: adding one turns an image that was merely observed into one that is checked.

A signing policy written once#

Who signs what a product runs is the same answer across every descriptor an organization owns, and copying it into each one means correcting it in each one.

# security/signers.saga-fragment.yaml
config:
  controls:
    provenance:
      signers:
        - name: platform-ci
          images: ["ghcr.io/acme/*"]
          github:
            repository: acme/ci-workflows
            workflow: .github/workflows/build-image.yml
            ref: refs/heads/main

# draugr.saga.yaml
fragments:
  - path: security/signers.saga-fragment.yaml
config:
  controls:
    provenance:
      enabled: true
      unmatched: warn     # the descriptor decides this, not the fragment

The descriptor's own signers, if it declares any, keep working: the fragment's are added after them. report.json records which file contributed them under descriptor.contributed.

Two signers matching one image is refused, so a fragment whose patterns overlap the descriptor's fails the descriptor loudly rather than quietly winning.

A monorepo serving several products#

The case fragments were designed around: one tree, many components, and products assembled from overlapping subsets of them.

services/payments/
    draugr.saga-fragment.yaml     # what payments is, everywhere
    azure.saga-fragment.yaml      # only when payments ships on Azure
    gcp.saga-fragment.yaml
services/ledger/
    draugr.saga-fragment.yaml
    azure.saga-fragment.yaml

# azure.saga.yaml
project: acme-azure
release: { version: "3.1.0" }
fragments:
  - path: "**/draugr.saga-fragment.yaml"
  - path: "**/azure.saga-fragment.yaml"

# gcp.saga.yaml
project: acme-gcp
release: { version: "3.1.0" }
fragments:
  - path: "**/draugr.saga-fragment.yaml"
  - path: "**/gcp.saga-fragment.yaml"

Two products over one tree, and adding a component is one new directory with no edit to either product's Saga.

Each product publishes to GitHub code scanning under its own category, taken from project:, so two pipelines uploading against one commit keep both sets of alerts. See publish to GitHub code scanning.

The same mechanics serve several teams rather than several products, and which of the two shapes a repository wants is the first decision in scanning a monorepo.

Name the audience in the stem#

This is the convention that makes it work. **/*.saga-fragment.yaml matches everything, including the other cloud's parts, and expressing "all of them except the GCP ones" needs glob subtraction, unreadable, and wrong the first time somebody adds a third cloud. Naming the audience in the file's stem means each product globs exact stems and never has to subtract.

Components merge rather than collide#

services/payments/draugr.saga-fragment.yaml:

components:
  - name: payments
    exposure: public
    criticality: critical
    repositories: [{ url: "services/payments" }]

services/payments/azure.saga-fragment.yaml:

components:
  - name: payments
    images: [{ image: "acme.azurecr.io/payments:3.1.0" }]

Same name, so these are one component with the repository and the image. The GCP product gets the same component with GCP surfaces instead.

Precedence#

exposure, criticality, builtBy and each label key take the value of the first file that sets them, in this order:

1. the root descriptor;
2. fragments: entries as written;
3. the files one pattern matches, in path order;
4. a fragment's own fragments:, immediately after it.

A fragment fills a field the root leaves unset and never replaces one it sets.

Set exposure and criticality in the root, or in the fragment listed first. If a component is more exposed on one cloud than another, leave them out of the root and the shared fragment, and set them in each variant.

Fragments from another repository#

fragments:
  - url: https://github.com/acme/platform.git
    revision: v2.4.0
    path: "components/**/draugr.saga-fragment.yaml"

revision is required. Without it, a fragment would track a moving branch and your gate could change with no commit in your own repository. A tag is fine, the commit it resolved to is recorded, so a tag that moves is visible afterwards.

Fragments inside a fetched repository resolve within it, so a repository that describes itself with relative paths keeps working when somebody else pulls it in.

The repository is cloned with its full history, so an exclusion in a fetched fragment carries that repository's git history in the report, read at the commit the revision resolved to.

Seeing what is actually in force#

Once a descriptor is assembled from parts, opening one file no longer answers the question. Here azure.saga.yaml also labels the component it ships:

# azure.saga.yaml
project: acme-azure
release: { version: "3.1.0" }
fragments:
  - path: "**/draugr.saga-fragment.yaml"
  - path: "**/azure.saga-fragment.yaml"
components:
  - name: payments
    labels: { cloud: azure }

draugr validate azure.saga.yaml --resolved

# Resolved Saga, every fragment merged. Generated by `draugr validate --resolved`.
# Valid input: comments are the provenance, so this can be scanned as it stands.
#
# root:     azure.saga.yaml
# fragment: services/payments/draugr.saga-fragment.yaml
# fragment: services/payments/azure.saga-fragment.yaml

project: acme-azure
release:
  version: 3.1.0
components:
  - name: payments
    labels:
      cloud: azure # from azure.saga.yaml
    exposure: public # from services/payments/draugr.saga-fragment.yaml
    criticality: critical # from services/payments/draugr.saga-fragment.yaml
    repositories:
      - url: services/payments
    images:
      - image: acme.azurecr.io/payments:3.1.0

The --resolved reference describes the comments.

The output is a valid descriptor, which is what makes it worth piping:

• Flatten it to cross an air gap. Resolve where there is network, carry the result across, scan it where there is none. See running air-gapped.
• Pin it. Every remote revision is recorded at the commit it resolved to, so a flattened copy is a lockfile in all but name.
• Diff it in CI. Commit the resolved descriptor and fail when it stops matching. A one-line fragments: change is then reviewed by its effect rather than by its cause.
• Query it. draugr validate azure.saga.yaml --resolved | yq '.components[].name'

Editor support#

Fragments are *.saga-fragment.yaml and have their own schema. A fragment has no release:, so checking one against the Saga's schema would report every valid fragment as broken.

# yaml-language-server: $schema=https://draugr.dev/schema/draugr.saga-fragment.schema.json

draugr schema --fragment prints the copy this build enforces, for pinning or air-gapped use.

Gotchas#

• A pattern that matches nothing is an error. Silence from a line somebody wrote on purpose is indistinguishable from a typo. If a product has no components on one cloud, do not list the pattern.
• Paths are relative to the file that names them, not to your working directory, so ../shared/exclusions.saga-fragment.yaml from a component's own fragment works.
• A file matched by two patterns is loaded once. Overlapping globs are normal, and loading a fragment twice would count its exclusions twice.