Skip to content

feat: unify server-side selector execution - #7185

Open
BillLeoutsakosvl346 wants to merge 23 commits into
stagingfrom
feat/unified-server-selector-execution
Open

feat: unify server-side selector execution#7185
BillLeoutsakosvl346 wants to merge 23 commits into
stagingfrom
feat/unified-server-selector-execution

Conversation

@BillLeoutsakosvl346

@BillLeoutsakosvl346 BillLeoutsakosvl346 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Moves every dynamic selector onto one exhaustive, server-authorized execution architecture while preserving the existing dropdown/combobox UX, React Query behavior, pagination, search, detail hydration, and saved workflow/connector formats.

  • Classifies all 93 dynamic selector keys in one browser-safe manifest:
    • 81 provider-server
    • 11 internal-server
    • 1 local (workspace.triggerTypes)
  • Executes all 92 server selectors through the selectors.execute application operation and POST /api/selectors/execute.
  • Resolves exact {{KEY}} references only after session and canonical workflow/workspace authorization.
  • Authorizes credential use and provider/service binding before provider work.
  • Returns only explicitly projected selector options; resolved context, tokens, credentials, and raw provider responses never cross the boundary.
  • Migrates workflow/block/trigger, connector, fork, search/replace, comparison, dynamic-option, and display-hydration consumers.
  • Deletes the complete browser provider-selector registry and the remaining 70 obsolete selector routes. The two CloudWatch listing routes and Confluence /page remain as thin compatibility adapters over staging's in-process operations; selectors do not call them.
  • Adds focused IMAP policy parity across selection, deployment validation, and polling.
  • Adds add-selector / validate-selector skills and updates adjacent integration, connector, trigger, block, and application-operation guidance.

There is no feature flag, browser-side provider fallback, provider-legacy classification, database migration, or saved-configuration migration.

Architecture

Before

flowchart TD
  A["Workflow, trigger, connector, fork, search, comparison, display"]
  --> B{"Surface-specific context builder"}
  B --> C["Browser personal-environment resolution"]
  B --> D["Unresolved dependency values"]
  C --> E["Client provider registry and provider-specific query key"]
  D --> E
  C --> X["Hidden shared value unavailable: dependency omitted or disabled"]
  E --> F["Provider-specific API route"]
  E --> G["Browser OAuth-token helper for some providers"]
  G --> F
  F --> H["External provider"]
Loading

After

flowchart TD
  A["Workflow, trigger, connector, fork, search, comparison, display"]
  --> B["One active-value context builder: literals plus exact unresolved references"]
  B --> C["One React Query facade with opaque privacy-safe identity"]
  C --> D["POST /api/selectors/execute"]
  D --> E["Authenticate session and resolve canonical scope"]
  E --> F["Authorize selectors.execute"]
  F --> G["Manifest capability and exact context allowlist"]
  G --> H["Resolve exact references server-side"]
  H --> I["Authorize credential use and provider/service binding"]
  I --> J["Apply destination policy and execute trusted adapter"]
  J --> K["Project, bound, sanitize, and return options only"]
  B --> L{"Local classification?"}
  L -->|"workspace.triggerTypes only"| M["Pure local attachment"]
Loading

Security boundary

  • Session authentication occurs before request-body parsing.
  • Workflow scope derives its workspace canonically; a supplied workspace ID is assertion-only and mismatch is concealed.
  • Selector keys, capabilities, context fields, credential services, provider scopes, operation kinds, and destinations are code-defined—not client-selected.
  • Only full-string environment references are resolved. Embedded interpolation is rejected.
  • Effective environment precedence remains workspace over personal; missing and inaccessible references share the same public error.
  • Credential existence, authorization, workspace membership, and provider mismatch share the same public connection error.
  • Query keys contain no context value, reference, credential ID, raw secret, resolved value, search/detail value, or value-derived hash.
  • The selector route is private, no-store; there is no selector result cache or high-volume selector audit event.
  • Protected values are tracked through execution and fail closed if projected option data contains protected plaintext.
  • Provider/SDK failures are caught and reduced to small typed public errors.
  • IMAP preserves literal/manual and viewable-reference behavior, but rejects hidden use-only username/password references before DNS. Host validation is DNS-pinned, TLS or mandatory STARTTLS is required, deployment/polling resolve just in time, and sensitive host validation suppresses hostname/IP/raw resolver details in logs.

Authorized use of a shared credential intentionally declassifies only the provider resource metadata represented by normalized options; it does not declassify the credential or referenced secret itself.

Explicit process-local token-cache exception

Selector data never enters browser state, query/cache keys, selector result caches, responses, logs, errors, audit metadata, or rate-limit keys. Authorized client-credential resolution may reuse the shared process-local token cache after authorization and provider binding.

This exception applies only to short-lived client-credential bearer tokens minted by the existing credential service. In selector privacy mode, credential-derived cache identities are HMACed. Each resolution reauthorizes the request, verifies provider binding, rereads the credential row, and validates the current encrypted-secret fingerprint before using a cached token, so credential rotation or reconnection prevents stale-token reuse. Entries remain in one server process, become ineligible for reuse at token expiry, are removed lazily on a later client-credential resolution or process exit, and never cross the selector boundary.

Explicit security-owner acceptance of this exception is required before merge. If it is not accepted, selector privacy mode must bypass client-credential token-cache reads and writes.

Migration inventory

  • 93 manifest keys = 92 exhaustive server attachments + 1 exhaustive local attachment.
  • 187 workflow/block/trigger declarations and 32 connector fields resolve to registered keys.
  • 33 browser provider modules and the shared browser selector infrastructure are removed.
  • 70 selector-only provider/API routes remain in the current diff and are removed; exact literal URL auditing found no deleted-route consumer.
  • CloudWatch log-group/log-stream routes and the Confluence page route remain as thin compatibility adapters over staging's in-process operations; selector adapters use their stricter server-only provider primitives directly.
  • lib/api/contracts/selectors now contains only the unified execute contract; retained CloudWatch, Confluence, and JSM tool contracts live under contracts/tools, and the Harmonic provider schema is adapter-local.
  • Diff against current staging: 330 files, 13,696 additions, 20,382 deletions.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: security architecture migration

Testing

Focused migration suites:

  • 20 test files, 191 tests passing across the route, executor, manifest, context builder, client facade, representative adapters, workflow search/display, OAuth privacy, and IMAP runtime policy.
  • Independent IMAP/security re-audit: 6 suites, 440 tests passing.
  • Independent completeness audit: 93/92/1 counts reconciled and every deleted route checked for literal consumers.
  • Independent client/UX audit: active registry-backed search, credential aliases, impersonation, and privacy-safe query revisions verified.
  • Review and staging-merge regressions: 20 focused files, 210 tests passing across environment ACLs, route concealment, context projection, query invalidation, provider binding, exact-reference sanitization, and retained CloudWatch/Confluence/JSM operation compatibility.
  • Final reference-hardening follow-up: 5 focused files, 39 tests passing across exact-reference collisions, requested-key environment resolution, and the shared testing mock.
  • Full repository run: all 19 workspace tasks passed; the Sim app completed 2,638 passing test files and 36,509 passing tests.

Repository gates:

  • bun run --cwd apps/sim type-check
  • bun run --cwd packages/testing type-check
  • bun run check:audits (all 37 audits)
  • bun run check:route-verbs
  • bun run check:api-validation:strict
  • bun run check:fork-dependent-coverage
  • bun run check:react-query
  • bun run check:client-boundary
  • bun run check:tool-registry-boundary
  • bun run check:skills
  • bun run lint:check (all 26 packages pass without warnings)
  • bun run --cwd apps/sim build
  • git diff --check

Review follow-up fixes

  • Active dependency projection now ignores dormant, noncanonical fields; Slack OAuth and bot-token mode switches cannot override one another.
  • Missing, mismatched, and inaccessible workflow/workspace scopes now share the same private, non-cacheable 404 response.
  • workspace.secretNames uses an ACL-aware names-only environment path that does not decrypt values or populate the decrypted snapshot cache.
  • OneDrive folder and SharePoint site selectors accept the exact Word/Excel credential aliases their blocks use, while file/list selectors remain narrowly bound.
  • Exact-reference detail hydration now sanitizes provider data with a detail-scoped exemption only for proven exact-ID repeats, then restores those repeats to the browser-owned {{KEY}}. The ID key / ID value collision is covered; partial or unrelated protected plaintext still fails closed.
  • Exact selector references now use a fresh ACL-aware requested-key resolver that decrypts only unique referenced values and neither reads nor warms the runtime snapshot cache.
  • Environment and credential mutations invalidate both normal selector queries and workflow search/replace selector queries through one privacy-safe helper.

Known limitation and deferred follow-ups

The review also identified adjacent pre-existing/shared concerns that are intentionally outside this PR’s locked scope:

  • Excel-backed SharePoint site discovery remains a known limitation: Microsoft Graph requires Sites.Read.All, while current Excel consent is file-only, so ordinary Excel credentials are expected to receive 403. The consent and credential model is tracked in #7197; no OAuth-scope or reconsent change is included here.
  • Thread authoritative custom and historical snapshot schemas through registry-independent selector context building.
  • Extend end-to-end cancellation into shared token minting and the OpenRouter catalog without breaking shared singleflight work.

Reviewer focus

  1. Authorization and resolution ordering in the central executor.
  2. Manifest/context readiness equivalence across workflow and connector surfaces.
  3. Credential provider binding and option projection in special adapters (Gmail/service accounts, Atlassian, Slack, Microsoft, Zoho, Snowflake/NetSuite, Managed Agent, CloudWatch).
  4. IMAP's deliberate hidden-auth restriction and deployment-actor behavior.
  5. Selector-only route deletion versus the retained runtime-shared routes.

Deferred live browser validation

The PR remains draft until disposable provider credentials are available. Automated tests do not require live credentials.

  • Gmail OAuth plus advanced credential-ID environment reference.
  • Jira credential plus domain environment reference.
  • CloudWatch raw access-key/secret/region references.
  • IMAP literal/manual success and hidden-use-only auth rejection.
  • One connector cascade (Google Drive or Airtable).
  • Optional search/pagination-heavy provider (Slack or Microsoft).

Excel-backed SharePoint site discovery is not an acceptance item for this PR and is expected to fail until #7197 selects and implements a consent model. Native SharePoint credential validation remains part of any Microsoft live-provider coverage.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • Live-provider browser checklist completed
  • Security owner accepts the process-local client-credential token-cache exception and its lazy-eviction behavior
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

No visual UI redesign. The selector UX is intentionally preserved; live-provider screenshots are deferred with the credential checklist above.

@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Aug 28, 2026 11:59pm

Request Review

# Conflicts:
#	apps/sim/app/api/tools/cloudwatch/describe-log-groups/route.ts
#	apps/sim/app/api/tools/cloudwatch/describe-log-streams/route.ts
#	apps/sim/app/api/tools/confluence/page/route.ts
#	apps/sim/app/api/tools/jsm/selector-requesttypes/route.ts
#	apps/sim/app/api/tools/jsm/selector-servicedesks/route.ts
#	apps/sim/lib/api/contracts/selectors/google.ts
#	apps/sim/lib/api/contracts/selectors/index.ts
#	apps/sim/lib/api/contracts/selectors/jira.ts
#	apps/sim/lib/api/contracts/selectors/microsoft.ts
#	apps/sim/lib/api/contracts/selectors/sharepoint.ts
#	apps/sim/lib/api/contracts/selectors/wealthbox.ts
#	apps/sim/lib/api/contracts/tools/index.ts
Bill Leoutsakos added 2 commits August 27, 2026 20:35
@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review (296 files, 100 file limit).

Bypass the limit by tagging @greptile-apps to review.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 296 files

You’re at about 90% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.

Re-trigger cubic

Comment thread apps/sim/blocks/blocks/microsoft_excel.ts Outdated
Comment thread apps/sim/lib/oauth/credential-service.ts Outdated
Comment thread apps/sim/lib/selectors/server/providers/confluence.ts
Comment thread apps/sim/lib/selectors/server/providers/pipedrive.ts
Comment thread apps/sim/lib/selectors/server/providers/snowflake.ts
Comment thread apps/sim/lib/selectors/server/providers/hubspot.ts
Comment thread apps/sim/lib/selectors/server/providers/hubspot.ts
Comment thread apps/sim/lib/selectors/server/providers/google.ts Outdated
Comment thread apps/sim/lib/selectors/server/providers/calcom.ts
Comment thread apps/sim/hooks/queries/selectors.test.tsx
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the addressed selector, OAuth, IMAP, provider-boundary, and UX findings; report only remaining or newly introduced issues.

@cubic-dev-ai

cubic-dev-ai Bot commented Aug 28, 2026

Copy link
Copy Markdown

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the addressed selector, OAuth, IMAP, provider-boundary, and UX findings; report only remaining or newly introduced issues.

@BillLeoutsakosvl346 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 314 files

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic

Comment thread apps/sim/lib/webhooks/deploy.ts
Comment thread apps/sim/app/api/webhooks/route.ts
Comment thread .agents/skills/add-integration/SKILL.md
Comment thread apps/sim/lib/selectors/server/sanitize.ts
Comment thread apps/sim/lib/selectors/server/providers/zoho-desk.ts
Comment thread apps/sim/lib/imap/connection.server.ts Outdated
Comment thread apps/sim/lib/selectors/server/providers/google.ts
Comment thread apps/sim/lib/selectors/server/providers/google.ts
Comment thread apps/sim/lib/selectors/server/sanitize.ts Outdated
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the addressed selector, OAuth, IMAP, provider-boundary, and UX findings; report only remaining or newly introduced issues.

@cubic-dev-ai

cubic-dev-ai Bot commented Aug 28, 2026

Copy link
Copy Markdown

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the addressed selector, OAuth, IMAP, provider-boundary, and UX findings; report only remaining or newly introduced issues.

@BillLeoutsakosvl346 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 315 files

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/lib/selectors/server/sanitize.ts Outdated
Comment thread apps/sim/app/api/webhooks/route.ts
Comment thread apps/sim/lib/selectors/server/providers/confluence.ts
Comment thread .agents/skills/add-block/SKILL.md Outdated
Bill Leoutsakos added 2 commits August 28, 2026 11:11
…r-selector-execution

# Conflicts:
#	apps/sim/app/api/tools/netsuite/objects/route.test.ts
#	apps/sim/app/api/tools/netsuite/objects/route.ts
#	scripts/check-tool-registry-boundary.baseline.json
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the repaired existing-IMAP deployment binding, connector selector validation guidance, and the documented Confluence and NetSuite dispositions. Report only remaining or newly introduced issues.

@cubic-dev-ai

cubic-dev-ai Bot commented Aug 28, 2026

Copy link
Copy Markdown

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the repaired existing-IMAP deployment binding, connector selector validation guidance, and the documented Confluence and NetSuite dispositions. Report only remaining or newly introduced issues.

@BillLeoutsakosvl346 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 315 files

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/app/api/webhooks/route.ts
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the existing-IMAP deployment binding update and rollback paths, including failed polling setup. Report only remaining or newly introduced issues.

@cubic-dev-ai

cubic-dev-ai Bot commented Aug 28, 2026

Copy link
Copy Markdown

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the existing-IMAP deployment binding update and rollback paths, including failed polling setup. Report only remaining or newly introduced issues.

@BillLeoutsakosvl346 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/app/api/webhooks/route.ts
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check all addressed selector, OAuth, IMAP, provider-boundary, and UX findings, including existing-IMAP deployment binding update and rollback paths. Report only remaining or newly introduced issues.

@cubic-dev-ai

cubic-dev-ai Bot commented Aug 28, 2026

Copy link
Copy Markdown

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check all addressed selector, OAuth, IMAP, provider-boundary, and UX findings, including existing-IMAP deployment binding update and rollback paths. Report only remaining or newly introduced issues.

@BillLeoutsakosvl346 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 315 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the IMAP active-deployment webhook lookup and rollback paths together with all previously addressed selector, OAuth, provider-boundary, and UX findings. Report only remaining or newly introduced issues.

@cubic-dev-ai

cubic-dev-ai Bot commented Aug 28, 2026

Copy link
Copy Markdown

@cubic-dev-ai rerun and review the complete updated PR against staging. Re-check the IMAP active-deployment webhook lookup and rollback paths together with all previously addressed selector, OAuth, provider-boundary, and UX findings. Report only remaining or newly introduced issues.

@BillLeoutsakosvl346 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 315 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

…r-selector-execution

# Conflicts:
#	apps/sim/hooks/selectors/providers/workspace/selectors.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant