Five defects the curacoach v1.0.0-qa10 report surfaced - #61
Merged
Conversation
Each one produced a confident wrong answer rather than an error, and each
asked someone for work that could not be done as written.
A remediation action dropped the npm scope. purl_name() took the last path
segment, so @nestjs/core, @sigstore/core and @strapi/core all became "core"
and one report carried three actions titled "upgrade core". The unit key is
built from that name, so two scoped packages sharing a last segment inside
one artifact would also have merged into a single action. Splitting on the
first @ was the same defect twice: an unencoded scoped purl answered "npm".
A prerelease was printed as the fix. The Fixed-in column pooled every fixed
version in a component group and took sorted(...)[-1]. multer's advisories
publish 2.2.0 and 3.0.0-alpha.2, @babel/core's publish 7.29.6 and 8.0.0-rc.6,
and string order puts the alpha last — so the column named a version a
released product cannot adopt while a stable fix stood beside it. A
prerelease is now dropped from the fix list wherever a stable one exists, and
the column picks per state. The new prerelease-only status covers the case
where an advisory publishes nothing but a prerelease; that case appears in
none of qa10's 64 advisories and is held by tests alone. Separate from
none-published because the answer differs: track the stable release rather
than carry a compensating control for good. Narrow by construction, since a
hyphen is not a prerelease marker in 31.1-jre or 2.36-9.
The same column also named too low a version. sorted() is lexical, so vite,
whose advisories publish 6.4.2 through 8.0.16 and 8.0.5, printed 8.0.5 —
below the version that carries the fix. Ordered numerically now.
An advisory without a purl aborted the whole scan. affected[].package.purl
is optional in OSV, and ("" | split("@")) is [] rather than [""] in jq, so
the index yielded null and startswith(null) ended the program. Latent today
because OSV populates the field, fatal for the entire run when it does not.
Staleness asked the same question every release. package:collection and
url_launcher are pinned by the Dart SDK constraint and can never look
current against a 12-month window, so every Flutter product recorded the
same reason again at every release. Answered by process default instead,
keyed on the pub.dev verified publisher, which is a fact from the registry.
npm has no equivalent: its author field is free text and never earns it.
The row stays in section 4 with its reason, since the staleness is real.
The backstop read the deployment history live whatever --now said, so a
report dated T could rest on a deployment made after T and the same evidence
gave different answers on different days.
Verified against the real registries, a real OSV advisory shape, the 7783
purls in the published documents, and the rendered PDF. Baseline 211 passed
on main; 227 offline and 233 with TEST_NETWORK=1 here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
grafele
force-pushed
the
soup-actionable-findings
branch
from
August 28, 2026 09:40
88090c1 to
feb0528
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while going through the dependency and vulnerability report for curacoach v1.0.0-qa10. Each one produced a confident wrong answer rather than an error, and each asked someone for work that could not be done as written.
An action dropped the npm scope
purl_name()returned the last path segment, so@nestjs/core,@sigstore/coreand@strapi/coreall becamecore. The qa10 report carried three separate actions titled "upgrade core".The worse half is latent: the unit key is
("dependency-upgrade", artifact, purl_name(purl)), so two scoped packages sharing a last segment inside one artifact merge into a single action carrying one of the two names. Nothing collided in that run because the three sat in different artifacts.Splitting on the first
@was the same defect again — an unencodedpkg:npm/@nestjs/core@11.1.14lost everything afterpkg:npm/and the function answerednpm.A prerelease was printed as the fix
The Fixed-in column pooled every fixed version in a component group and took
sorted(...)[-1]. Both advisories in question publish a stable fix and a prerelease, and lexical order puts the prerelease last:So the column named a version a released medical device cannot adopt while the stable fix stood next to it.
scan-vulns.shnow drops a prerelease from the fix list wherever a stable one exists, and the column selects per state rather than from a pool.New status
prerelease-onlyfor the case the pool hid: an advisory that publishes nothing but a prerelease. Upstream has a fix, it is simply not adoptable yet, and the work is to track the stable release rather than to carry a compensating control indefinitely — which is why it stays separate fromnone-published. That case occurs in none of qa10's 64 advisories; it is held by tests, not by a finding in this report.Detection is narrow on purpose — only semver ecosystems, only recognised prerelease tokens. A hyphen alone is not one:
3.0.0-alpha.28.0.0-rc.631.1-jre2.36-9v0.0.0-20240101-abcAn unrecognised token stays
available, so the failure direction is to keep offering an upgrade rather than to withdraw a real one.The same column named too low a version
sorted()is lexical, so8.0.5sorts above8.0.16. Real case in this report:Understating the required version is the worse direction of the two, so the column is ordered numerically now. Pre-existing, but on the lines this PR rewrites anyway.
An advisory without a purl aborted the whole scan
affected[].package.purlis optional in the OSV schema, and in jq("" | split("@"))is[]rather than[""]. The index yieldednullandstartswith(null)ended the entire program — one such advisory fails the whole run, not just its own match.Latent today: every advisory I sampled populates the field. Found because the test fixtures for the item above did not.
Staleness asked the same question at every release
package:collectionis published bydart.devandurl_launcherbyflutter.dev. Both are pinned by the Dart SDK constraint and can never look current against a 12-month window, so every Flutter product recorded the same reason again at every release — the outcome WI-006-09 exists to remove.Now answered by process default,
dependency_currency.stale_exempt_publishers, keyed on the pub.dev verified publisher. That is a fact read from the registry: pub.dev proves domain ownership before it shows one. npm has no equivalent — itsauthorfield is free text set by whoever publishes, andtslibreports "Microsoft Corp." — so no npm package is exempt and a test holds that.Three properties worth naming:
verified publisher dart.devand its reason instead of "No decision recorded.", unshaded and out of the summary tile. The staleness is real and stays visible.validate-policy.shrefuses a publisher beyond the default withoutdependency_currency.reason, same asmax_behind.google.devis deliberately not on the list: it publishes a grab-bag rather than the SDK, andvisibility_detector(last release 2023) is exactly the finding a product should still have to answer. On the eight stale pub packages in qa10 that leaves six open and pre-answers two.The backstop read releases from after the report date
production_deploys()queried GitHub live whatever--nowsaid. A report dated T could claim a cadence that only holds because of a deployment made after T, and the same evidence directory gave different answers on different days. That is what madenet_backstop_cadence_counts_production_releases_onlygo red on its own — alvie deployed on 2026-08-12 and a run frozen at 2026-08-02 counted it.In a live run
not_afteris the moment the run started, so nothing is excluded that the run could have known about. A bound that removes everything reports "no records", never a falsebroken.Testing
17 new tests. Baseline on
mainis 211 passed; this branch is 227 offline, 233 withTEST_NETWORK=1, 0 failures either way.Verified beyond the suite:
purl_nameagainst every purl shape, and against all 7783 purls in the published documents: no defect, and the 27 scoped packages that qa10 collapsed tocorenow keep their namespace.Fix availability: … 1 only as a prerelease,Fixed in: 3.0.0-alpha.2 / prerelease only, the exemption row in section 4, and three correctly named actions in section 5.QuickBirdEng/alviedeployment history, frozen and live.Section 4's column widths were rebalanced within the same 170mm — Status now carries a sentence and Registry status a publisher domain, both were sized for two words.
Note for whoever runs the suite locally:
classify_*andgrq4_*needpip install cvsssince #57, otherwise 33 tests fail before any of this.🤖 Generated with Claude Code