Skip to content

Five defects the curacoach v1.0.0-qa10 report surfaced - #61

Merged
grafele merged 1 commit into
mainfrom
soup-actionable-findings
Aug 28, 2026
Merged

Five defects the curacoach v1.0.0-qa10 report surfaced#61
grafele merged 1 commit into
mainfrom
soup-actionable-findings

Conversation

@grafele

@grafele grafele commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Found while going through the dependency and vulnerability report for curacoach v1.0.0-qa10. Each one produced a confident wrong answer rather than an error, and each asked someone for work that could not be done as written.

An action dropped the npm scope

purl_name() returned the last path segment, so @nestjs/core, @sigstore/core and @strapi/core all became core. The qa10 report carried three separate actions titled "upgrade core".

The worse half is latent: the unit key is ("dependency-upgrade", artifact, purl_name(purl)), so two scoped packages sharing a last segment inside one artifact merge into a single action carrying one of the two names. Nothing collided in that run because the three sat in different artifacts.

Splitting on the first @ was the same defect again — an unencoded pkg:npm/@nestjs/core@11.1.14 lost everything after pkg:npm/ and the function answered npm.

before                              after
pkg:npm/%40nestjs/core@11.1.14   -> core     @nestjs/core
pkg:npm/%40sigstore/core@2.0.0   -> core     @sigstore/core
pkg:npm/@nestjs/core@11.1.14     -> npm      @nestjs/core
pkg:maven/com.foo/bar@1.0        -> bar      com.foo/bar

A prerelease was printed as the fix

The Fixed-in column pooled every fixed version in a component group and took sorted(...)[-1]. Both advisories in question publish a stable fix and a prerelease, and lexical order puts the prerelease last:

multer 2.0.2       CVE-2026-5038 / -5079   fixed: 2.2.0, 3.0.0-alpha.2
@babel/core 7.29.0 CVE-2026-49356          fixed: 7.29.6, 8.0.0-rc.6

So the column named a version a released medical device cannot adopt while the stable fix stood next to it. scan-vulns.sh now drops a prerelease from the fix list wherever a stable one exists, and the column selects per state rather than from a pool.

New status prerelease-only for the case the pool hid: an advisory that publishes nothing but a prerelease. Upstream has a fix, it is simply not adoptable yet, and the work is to track the stable release rather than to carry a compensating control indefinitely — which is why it stays separate from none-published. That case occurs in none of qa10's 64 advisories; it is held by tests, not by a finding in this report.

Detection is narrow on purpose — only semver ecosystems, only recognised prerelease tokens. A hyphen alone is not one:

version ecosystem prerelease
3.0.0-alpha.2 npm yes
8.0.0-rc.6 npm yes
31.1-jre Maven no (guava)
2.36-9 Debian no
v0.0.0-20240101-abc Go no

An unrecognised token stays available, so the failure direction is to keep offering an upgrade rather than to withdraw a real one.

The same column named too low a version

sorted() is lexical, so 8.0.5 sorts above 8.0.16. Real case in this report:

vite   stable fixes: 6.4.2, 6.4.3, 7.3.2, 7.3.5, 8.0.16, 8.0.5
       printed: 8.0.5        the version that carries the fix: 8.0.16

Understating the required version is the worse direction of the two, so the column is ordered numerically now. Pre-existing, but on the lines this PR rewrites anyway.

An advisory without a purl aborted the whole scan

affected[].package.purl is optional in the OSV schema, and in jq ("" | split("@")) is [] rather than [""]. The index yielded null and startswith(null) ended the entire program — one such advisory fails the whole run, not just its own match.

Latent today: every advisory I sampled populates the field. Found because the test fixtures for the item above did not.

Staleness asked the same question at every release

package:collection is published by dart.dev and url_launcher by flutter.dev. Both are pinned by the Dart SDK constraint and can never look current against a 12-month window, so every Flutter product recorded the same reason again at every release — the outcome WI-006-09 exists to remove.

Now answered by process default, dependency_currency.stale_exempt_publishers, keyed on the pub.dev verified publisher. That is a fact read from the registry: pub.dev proves domain ownership before it shows one. npm has no equivalent — its author field is free text set by whoever publishes, and tslib reports "Microsoft Corp." — so no npm package is exempt and a test holds that.

Three properties worth naming:

  • It suppresses nothing. The row stays in section 4 with verified publisher dart.dev and its reason instead of "No decision recorded.", unshaded and out of the summary tile. The staleness is real and stays visible.
  • It answers staleness only. A listed publisher's package that is behind the update limit is still behind. Deprecation also outranks it.
  • Extending it is a widening. validate-policy.sh refuses a publisher beyond the default without dependency_currency.reason, same as max_behind.

google.dev is deliberately not on the list: it publishes a grab-bag rather than the SDK, and visibility_detector (last release 2023) is exactly the finding a product should still have to answer. On the eight stale pub packages in qa10 that leaves six open and pre-answers two.

The backstop read releases from after the report date

production_deploys() queried GitHub live whatever --now said. A report dated T could claim a cadence that only holds because of a deployment made after T, and the same evidence directory gave different answers on different days. That is what made net_backstop_cadence_counts_production_releases_only go red on its own — alvie deployed on 2026-08-12 and a run frozen at 2026-08-02 counted it.

In a live run not_after is the moment the run started, so nothing is excluded that the run could have known about. A bound that removes everything reports "no records", never a false broken.

Testing

17 new tests. Baseline on main is 211 passed; this branch is 227 offline, 233 with TEST_NETWORK=1, 0 failures either way.

Verified beyond the suite:

  • purl_name against every purl shape, and against all 7783 purls in the published documents: no defect, and the 27 scoped packages that qa10 collapsed to core now keep their namespace.
  • Prerelease detection against the guava / Debian / Go pseudo-version cases above.
  • pub.dev's publisher endpoint against the live API for all eight stale packages in the qa10 report.
  • The rendered PDF, not just the JSON: Fix availability: … 1 only as a prerelease, Fixed in: 3.0.0-alpha.2 / prerelease only, the exemption row in section 4, and three correctly named actions in section 5.
  • The backstop bound against the real QuickBirdEng/alvie deployment history, frozen and live.

Section 4's column widths were rebalanced within the same 170mm — Status now carries a sentence and Registry status a publisher domain, both were sized for two words.

Note for whoever runs the suite locally: classify_* and grq4_* need pip install cvss since #57, otherwise 33 tests fail before any of this.

🤖 Generated with Claude Code

Each one produced a confident wrong answer rather than an error, and each
asked someone for work that could not be done as written.

A remediation action dropped the npm scope. purl_name() took the last path
segment, so @nestjs/core, @sigstore/core and @strapi/core all became "core"
and one report carried three actions titled "upgrade core". The unit key is
built from that name, so two scoped packages sharing a last segment inside
one artifact would also have merged into a single action. Splitting on the
first @ was the same defect twice: an unencoded scoped purl answered "npm".

A prerelease was printed as the fix. The Fixed-in column pooled every fixed
version in a component group and took sorted(...)[-1]. multer's advisories
publish 2.2.0 and 3.0.0-alpha.2, @babel/core's publish 7.29.6 and 8.0.0-rc.6,
and string order puts the alpha last — so the column named a version a
released product cannot adopt while a stable fix stood beside it. A
prerelease is now dropped from the fix list wherever a stable one exists, and
the column picks per state. The new prerelease-only status covers the case
where an advisory publishes nothing but a prerelease; that case appears in
none of qa10's 64 advisories and is held by tests alone. Separate from
none-published because the answer differs: track the stable release rather
than carry a compensating control for good. Narrow by construction, since a
hyphen is not a prerelease marker in 31.1-jre or 2.36-9.

The same column also named too low a version. sorted() is lexical, so vite,
whose advisories publish 6.4.2 through 8.0.16 and 8.0.5, printed 8.0.5 —
below the version that carries the fix. Ordered numerically now.

An advisory without a purl aborted the whole scan. affected[].package.purl
is optional in OSV, and ("" | split("@")) is [] rather than [""] in jq, so
the index yielded null and startswith(null) ended the program. Latent today
because OSV populates the field, fatal for the entire run when it does not.

Staleness asked the same question every release. package:collection and
url_launcher are pinned by the Dart SDK constraint and can never look
current against a 12-month window, so every Flutter product recorded the
same reason again at every release. Answered by process default instead,
keyed on the pub.dev verified publisher, which is a fact from the registry.
npm has no equivalent: its author field is free text and never earns it.
The row stays in section 4 with its reason, since the staleness is real.

The backstop read the deployment history live whatever --now said, so a
report dated T could rest on a deployment made after T and the same evidence
gave different answers on different days.

Verified against the real registries, a real OSV advisory shape, the 7783
purls in the published documents, and the rendered PDF. Baseline 211 passed
on main; 227 offline and 233 with TEST_NETWORK=1 here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@grafele
grafele force-pushed the soup-actionable-findings branch from 88090c1 to feb0528 Compare August 28, 2026 09:40
@grafele grafele changed the title Four defects the curacoach v1.0.0-qa10 report surfaced Five defects the curacoach v1.0.0-qa10 report surfaced Aug 28, 2026
@grafele
grafele merged commit 48b2ee6 into main Aug 28, 2026
@grafele
grafele deleted the soup-actionable-findings branch August 28, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant