ci: scan issue and comment bodies — this repo has never scanned one - #27
ci: scan issue and comment bodies — this repo has never scanned one#27yakimoto wants to merge 16 commits into
Conversation
…ment body Measured across all 28 public wave-av repos (claude-workstation#1747, #1794): TWO coverage shapes satisfy the one required check name `Secrets + content policy`. 27 repos triggers: pull_request, push, workflow_dispatch jobs: guard 1 repo triggers: + issues, issue_comment jobs: + body-guard This repo is in the 27. All 28 report the same green check. The outlier is wave-moq-edge, and its own comment says why it matters: "`edited` matters as much as `opened`: a body can be made to leak long after the PR is first raised, and until this workflow covered it, nothing ever re-scanned." A PR/issue/comment BODY is exactly as world-readable as the tree, and until now it was scanned by nothing server-side. That gap was not theoretical on wave-moq-edge: a PR was blocked for naming a private repo in wrangler.toml while the very same name, with more operational detail attached, sat unchallenged in its body. WHAT LANDS HERE — the bundle the workflow's own header names, minus what this repo already has (.gitleaks.toml and content-policy.sh are already vendored): .github/workflows/public-repo-guard.yml replaced (73 -> 163 lines) scripts/public-repo-guard/body-policy.sh new, mode 100755 scripts/public-repo-guard/tests/body-policy.test.sh new, mode 100755 Copied from wave-moq-edge, which has run this shape in production. Modes preserved via the git trees API — the contents API would have created both scripts 100644. HONEST ABOUT WHAT IT CAN DO. On a PR this PREVENTS the merge. On an issue or comment the text is already public the moment it posts, so this is DETECTION: it says go redact, fast. Only a client-side pre-write hook stops that class before publication. Also inherited from the reference: concurrency moves from workflow-level to PER JOB, because the two jobs want opposite behaviour. A workflow-level group forced one policy on both, and rapid body edits cancelled the tree job repeatedly — every cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while the live runs were green. The body gate ships with its own fixtures and runs them in CI. Its NEGATIVE cases are the load-bearing half: a leak gate that blocks legitimate cross-repo references gets switched off, and then it protects nothing. Refs wave-av/claude-workstation#1747. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto reviews are limited based on label configuration. 🚫 Excluded labels (none allowed) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_31b04dac-ddcf-4755-b62e-bd5e19d92c8d) |
ApprovabilityVerdict: Not approved Macroscope's review found this PR not approvable — This PR adds a substantial security-sensitive CI gate that scans public conversation content and can block merges, while also changing required-check and merge-queue behavior. Unresolved comments raise a possible scanner bypass and an installation/event-lifecycle failure mode, so the workflow and policy behavior need human review. Not approved because:
Review your spending limits in Billing settings. You can add or adjust custom eligibility rules. Learn more. |
PR Summary by QodoCI: scan issue/PR/comment bodies in public-repo-guard (with fixtures + safe concurrency)
AI Description
Diagram
High-Level Assessment
Files changed (3)
|
Code Review by Qodo
1. PR-controlled body scanner
|
Qodo Fixer✅ Merged (0) · ☑ Fixed (0) Process
|
Three review findings on the body gate: - ABOUT_THE_CONTROL exempted every rule, so a real credential on a line that mentioned the policy scanned clean. The allowlist is now opt-in per rule (prose flag) and only the heuristic rules (internal-marker, private-repo-ops) honour it; credential/infra formats always block. - A global (?i) leaked onto the SCREAMING_CASE credential-name branch of private-repo-ops, so lowercase code talk (session_token) near a repo name blocked ordinary prose. Case-insensitivity is now scoped to the repo names and the English phrase alternatives only. - Both policy scripts require rg -P, but Ubuntu's apt ripgrep is built without PCRE2, which would turn the required check permanently red. Both jobs now install the upstream binary pinned + SHA-256-verified (same pattern as gitleaks), and body-policy.sh refuses a PCRE2-less rg up front with a legible error. Fixtures added for all three regressions; suite passes 26/26. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…names Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…sede a failing one Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
… scan cannot match it Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…after the repo name Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…for the tree job Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…ed review comments are all scanned Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Review events (pull_request_review, pull_request_review_comment) run in the PR's context and publish a check run on the PR head SHA; with the guard job skipping them, that run lands as 'skipped', which GitHub treats as passing while evaluating the most recent check run per name, so any review comment could supersede a failing tree scan with a green rubber stamp. The guard job now re-runs the tree scan on those events, same as pull_request: edited. body-policy.sh now fails closed (exit 2) in CI when GUARD_PRIVATE_REPOS is empty or contains no names: a missing or renamed org variable must go red, not silently skip the private-repo proximity rule and report a pass over an unscanned leak class. Local runs still skip the rule, and both behaviours are pinned by new fixtures. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_86adf5e8-be25-4e09-b0ce-5d6efff24718) |
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Addresses a review finding (qodo-code-review, PR#27): the body-guard job checked out the tree with no ref, defaulting to refs/pull/N/merge on pull_request events -- the PRs OWN tree. A malicious PR could edit scripts/public-repo-guard/body-policy.sh in the same PR to always pass, defeating the body-leak gate. Pin to the PR base sha (untouched by the PR) when one exists, else the default branch (issues/ issue_comment already run in the default-branch context).
🤖 CodeAnt AI — Review Status
|
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_a225b54d-5325-4c8b-b2cc-cb30549210dd) |
|
ⓘ Qodo reviews are paused because your workspace is out of credits. Ask your workspace admin to add credits to resume reviews. Manage billing |
| # the PR) when one exists; otherwise (issues/issue_comment, which | ||
| # already run in the default-branch context) fall back to the | ||
| # default branch, which is equally trusted. | ||
| ref: ${{ github.event.pull_request.base.sha || github.event.repository.default_branch }} |
There was a problem hiding this comment.
Suggestion: The body job checks out the PR base, so the installation PR's base may lack body-policy.sh; the later command then fails before scanning the body. [state/lifecycle]
Assessment: 🟠 Major · 🔁 Occurrence: Often
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** .github/workflows/public-repo-guard.yml
**Line:** 203:203
**Comment:**
*State Lifecycle: The body job checks out the PR base, so the installation PR's base may lack `body-policy.sh`; the later command then fails before scanning the body.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| if [[ "$scope" == "prose" ]]; then | ||
| matches="$(printf '%s' "$matches" | rg -vNiP -- "$ABOUT_THE_CONTROL" || true)" | ||
| fi |
There was a problem hiding this comment.
Suggestion: The prose filter removes every heuristic match on a line mentioning body-policy or another control name, allowing leaked operational details to bypass detection by appending that phrase. [security]
Assessment: 🔴 Critical · 🔁 Occurrence: Sometimes
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** scripts/public-repo-guard/body-policy.sh
**Line:** 68:70
**Comment:**
*Security: The prose filter removes every heuristic match on a line mentioning `body-policy` or another control name, allowing leaked operational details to bypass detection by appending that phrase.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
CodeAnt Nitpicks1 code suggestion1. The harness checks only the exit code and complete-body absence, so another rule can mask a broken rule or a partial secret can still leak in annotations.Code quality · |
The previous commit pinned body-guards checkout to the PR base sha, but THIS PR is what first adds scripts/public-repo-guard/body-policy.sh -- base (main, pre-merge) does not have it yet, so the job 404d on its own script (observed: two red "Body content policy" runs on this PR after the pin landed). Add a second, unpinned checkout that only runs when hashFiles finds the script missing at the trusted ref -- true only for this bootstrap PR, never for a later PR trying to tamper with an already-merged script.
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_7b84e0f1-ac5f-4c09-bf72-9d596529c4e0) |
…he bootstrap deadlock This branch introduced the body scanner as a second job inside public-repo-guard.yml, pinned to a trusted ref and refusing to execute the PR own copy. That refusal is a bootstrap deadlock: the PR that introduces the scanner can never satisfy a check that will only run a copy already merged to the base. A later generation of exactly this change has since landed on main in two sibling public repos; this commit ports that shape rather than inventing a third variant. Three differences from what this branch had: 1. File-level split. The body scan moves to its own workflow file, .github/workflows/public-repo-guard-body.yml. public-repo-guard.yml now triggers only on tree-changing events, so a comment or review event no longer publishes a skipped check-run under the REQUIRED tree-scan name. Ruleset evaluation reads the newest run of a name and treats skipped as passing, so a chatty thread could mask a failed or never-completed tree verdict. A job-level if did not close that; removing the trigger does. 2. No trusted-copy dance. The body job runs the repo checked-out copy directly. It triggers on pull_request, never pull_request_target, so a fork PR gets no write token and no repo secrets, and the untrusted title and body are read out of the event payload file with jq into another file, never interpolated into a run block and never passed through an environment variable. The deadlock disappears because the gate no longer needs a pre-merged copy to be safe. 3. ripgrep pinned and SHA-256 verified in both jobs. The rules are rg -P and the apt package is built without PCRE2, which would fail every rule with exit 2 rather than a verdict. Two hardenings kept that only one landed copy carried, so this port is not weaker than either parent: - GUARD_PRIVATE_REPOS is normalised for newlines and carriage returns before splitting. read stops at the first newline, so a newline-separated value configured only the first name and reported a pass over the rest, and a CRLF value glued an invisible carriage return to every name so the built pattern matched nothing and the rule failed open silently. - An empty GUARD_PRIVATE_REPOS fails CLOSED in CI (exit 2) instead of warning. A missing or renamed variable means the flagship rule scanned nothing while the job reports green, which is the rubber stamp every other stage in this script refuses. Local runs still skip it. Both carry fixture regressions. The suite runs from the tree job and is 45 green locally, including the fail-closed cases for a broken filter stage, a missing argument, and an unconfigured variable. Fixture repo names stay synthetic: this file is public. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_f204d436-c75b-40c2-82e4-6f3f80d24b7b) |
User description
This repo's
public-repo-guardhas never scanned a single issue or comment body.Measured across all 28 public wave-av repos (
wave-av/claude-workstation#1747,#1794): two coverage shapes satisfy the one required check nameSecrets + content policy.pull_request, push, workflow_dispatchguardissues,issue_commentbody-guardThis repo is in the 27. All 28 report the same green check — because a required check asserts that something named X passed, never what X examined.
The outlier is
wave-moq-edge, and its own comment says why it matters:That gap was not theoretical there: a PR was blocked for naming a private repo in
wrangler.tomlwhile the very same name, with more operational detail attached, sat unchallenged in its body.What lands
Three files — the bundle the workflow's own header names, minus what this repo already has (
.gitleaks.tomlandcontent-policy.share already vendored, and are checked as prerequisites; a repo missing either is refused rather than half-installed):The workflow's header names four files as the install unit but executes a fifth —
tests/body-policy.test.sh, in its own self-test step. Omitting it installs a workflow that fails on a step nobody read, so the manifest ships it. Modes are preserved via the git trees API; the contents API creates100644regardless, which would silently break running these scripts as executables.Planned by
governance/lib/vendor-bundle.mjs(claude-workstation#1850) against a checked-in manifest, not by ad-hoc shell.One deliberate divergence from the reference, stated rather than silent
The shipped workflow is
wave-moq-edge's withactions/checkoutbumped from v5.0.1 to v7.0.1 (3d3c42e5aac5ba805825da76410c181273ba90b1), the pin already used byclaude-workstation's own gate.Copying verbatim was checked first and rejected on evidence: of the 18 target repos, 17 carry a byte-identical guard, and
wave-realtime-edgealready runs v7.0.0 — so a verbatim copy would have downgraded it, and shipped a stale pin to the other 17. A separate PR brings the reference itself up to the same pin.Honest about what this can and cannot do
On a PR this PREVENTS the merge. On an issue or comment the text is already public the moment it posts, so this is DETECTION: it says go redact, fast. Only a client-side pre-write hook stops that class before publication.
Also inherited from the reference: concurrency moves from workflow-level to per job, because the two jobs want opposite behaviour. A workflow-level group forced one policy on both, and rapid body edits cancelled the tree job repeatedly — every cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while the live runs were green.
The body gate ships with its own fixtures and runs them in CI. Its negative cases are the load-bearing half: a leak gate that blocks legitimate cross-repo references gets switched off, and then it protects nothing.
Refs
wave-av/claude-workstation#1747.Note
Medium Risk
Changes required-status CI behavior and merge gating on PR text; misconfigured
GUARD_PRIVATE_REPOSor scanner failures fail closed and can block merges until fixed.Overview
Closes a gap where only the published tree was scanned — PR/issue/comment/review text is now checked server-side via a new
body-policy.shand a separatepublic-repo-guard-body.ymlworkflow that materializes event payloads safely and runs the policy on titles/bodies (includingeditedevents).The tree workflow is reworked so the required "Secrets + content policy" check no longer runs (or publishes
skipped) on comment/review events — a file-level split that fixes branch-protection masking when skipped runs overwrote real failures. The tree job also gainsmerge_group, per-job concurrency,actions/checkoutv7.0.1,persist-credentials: false, and a pinned PCRE2 ripgrep install (replacing aptrgthat broke-Prules). Fixture tests for body policy run from the tree job in CI.body-policy.shmirrors many tree leak rules but tunes them for prose (e.g. private-repo names only block when paired with operational detail on the same line, redacted annotations, fail-closed on scanner/emptyGUARD_PRIVATE_REPOSin CI).body-policy.test.shadds must-block, must-pass, and fail-closed coverage.Reviewed by Cursor Bugbot for commit 882e965. Bugbot is set up for automated code reviews on this repo. Configure here.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is enabled.Note
Scan PR/issue/comment/review bodies for credential and infrastructure leaks in CI
body-guardCI job in public-repo-guard.yml triggered byissues,issue_comment,pull_request_review, andpull_request_review_commentevents, with per-event concurrency keyed to the most specific object ID.guardjob triggers to include review events and body-policy fixture tests; adds PCRE2-capable ripgrep installation to prevent false scanner failures.GUARD_PRIVATE_REPOSis unset in CI or when a non-PCRE2rgis detected, blocking the check run.Macroscope summarized c284c89.
Summary by Sourcery
Expand the public repository guard to scan all public conversation text while preserving reliable, merge-blocking tree checks.
New Features:
Bug Fixes:
Enhancements:
Build:
CI:
Tests:
CodeAnt-AI Description
Scan public PR, issue, and comment text for internal leaks
What Changed
Impact
✅ Fewer internal leaks in public PRs and discussions✅ Edited comments and reviews are rescanned✅ Required tree-scan results cannot be masked by skipped body events💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.