Skip to content

Add App Doctor scan command - #8429

Draft
jplhomer wants to merge 1 commit into
mainfrom
app-doctor/cli-scan
Draft

Add App Doctor scan command#8429
jplhomer wants to merge 1 commit into
mainfrom
app-doctor/cli-scan

Conversation

@jplhomer

Copy link
Copy Markdown
Contributor

Problem

Shopify app developers need a Shopify-specific security review workflow that combines deterministic checks with structured prompts for deeper review by their coding agent.

Solution

Add shopify app doctor scan directly to Shopify CLI.

The command:

  • scans app configuration and source for Shopify-specific security issues;
  • writes app-doctor-review.json with 16 versioned semantic checks;
  • writes a schema-versioned, unsigned app-doctor-trace.json;
  • accepts --findings <file> to validate agent findings and compile them into the local trace;
  • supports human and JSON output plus severity-based CI blocking;
  • redacts matched secrets from findings, evidence, output, and traces;
  • reports the CLI engine and ruleset versions;
  • optionally shows AI Toolkit skill setup instructions without modifying agent configuration.

The engine lives inside @shopify/app; it is not a separate package or runtime dependency. Semantic prompts are embedded in the CLI bundle while adjacent Markdown remains their source of truth.

shopify app doctor submit is deliberately not implemented here. That command is reserved for future authenticated upload to Shopify.

Risk

This adds a substantial local scanner and a new trace schema. Findings are informative, and trace v1 is explicitly unsigned. Static analysis cannot prove an app secure; skipped files and incomplete coverage are recorded rather than treated as clean.

The command does not reuse the unrelated DoctorSuite release-test harness.

Top-hatting

pnpm install --frozen-lockfile
pnpm --filter @shopify/app type-check
pnpm --filter @shopify/app lint
pnpm --filter @shopify/app build
pnpm nx bundle cli --skip-nx-cache

Run a scan:

node packages/cli/bin/run.js app doctor scan /path/to/app --skip-skill
CI=1 node packages/cli/bin/run.js app doctor scan /path/to/app --json --skip-skill

After completing the generated review pack:

node packages/cli/bin/run.js app doctor scan /path/to/app \
  --findings /path/to/app/findings.json \
  --skip-skill

Verification

  • App typecheck and lint pass.
  • App and CLI builds pass.
  • CLI bundle and Oclif manifest generation pass.
  • Focused engine/command/service suite: 12 files, 99 tests passing.
  • CLI registration test passes.
  • commands --tree snapshot passes and exposes app doctor scan.
  • Bundled CLI smoke test produced parseable JSON, a 16-check review pack, and a schema v1 trace containing all 33 registry entries.

@github-actions github-actions Bot added the Area: @shopify/app @shopify/app package issues label Aug 28, 2026
const direct = RUBY_REQUEST_SOURCE.test(valueExpression) || JS_REQUEST_SOURCE.test(valueExpression)
const visible = boundByLine[i] ?? new Set<string>()
const viaLocal = [...visible].some((name) =>
new RegExp(`^${name.replace(/[@$]/g, '\\$&')}\\b`).test(valueExpression),
@jplhomer

Copy link
Copy Markdown
Contributor Author

/snapit

@github-actions

Copy link
Copy Markdown
Contributor

🫰✨ Thanks @jplhomer! Your snapshot has been published to npm.

Test the snapshot by installing your package globally:

pnpm i -g --@shopify:registry=https://registry.npmjs.org @shopify/cli@0.0.0-snapshot-20260828224943

Caution

After installing, validate the version by running shopify version in your terminal.
If the versions don't match, you might have multiple global instances installed.
Use which shopify to find out which one you are running and uninstall it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: @shopify/app @shopify/app package issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants