Skip to content

feat(tables): add automatic row expiration with TTL - #7071

Merged
j15z merged 19 commits into
refactor/table-column-type-extension-pointsfrom
feat/table-row-ttl
Aug 29, 2026
Merged

feat(tables): add automatic row expiration with TTL#7071
j15z merged 19 commits into
refactor/table-column-type-extension-pointsfrom
feat/table-row-ttl

Conversation

@j15z

@j15z j15z commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Tables can now expire rows automatically with one Expiration column. Cells use the date editor, APIs and workflows store integer Unix epoch seconds, and a bounded cleanup job removes eligible rows every fifteen minutes.

This is PR 2 of 4 in the row-expiration stack. It builds on the generic registry hooks in PR #7119; PR #7072 supplies the safety-critical timezone consolidation; PR #7161 lets deletions trigger workflows.

The global table-row-ttl runtime feature flag gates UI availability, direct schema mutations, workflow-created columns, and cleanup dispatch. Outside AppConfig, the TABLE_ROW_TTL secret is the global on/off fallback. TTL-specific CSV coercion stays in a lightweight column-type capability map so shared workspace pages do not load the full registry.

Cleanup is round-robin across candidate tables and is capped at 100 batches of 500 rows per run. Each table is scanned by created_at and id so older rows cannot starve, but only the Expiration value decides whether a row is deleted. The supporting database index matches that scan order.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: ___________

Testing

  • Focused coverage exercises the feature flag, every schema-ingress guard, schema limits, import coercion, cleanup behavior, cron dispatch, and column pickers.
  • Cleanup coverage verifies batching and fairness while future, empty, and malformed Expiration values remain untouched.
  • The cron route coverage verifies authentication, fifteen-minute deduplication, feature-disabled behavior, and async-job dispatch.
  • At the restacked stack tip, 29 focused test files pass with 453 tests.
  • bun run type-check, bun run lint:check, bun run check:api-validation, bun run check:tool-registry-boundary, and bun run check:migrations origin/staging pass.
  • Review focus: deletion eligibility must depend only on the stored Expiration epoch; a row may be removed late, but never early.

Known Residuals

  • The cleanup predicate reads a dynamic JSONB Expiration key, so it cannot use a per-column expiration range index. Validate the plan with production-scale EXPLAIN ANALYZE; use an indexed expiration-candidate relation if large-table cleanup cost is too high.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

No screenshots captured. Component tests cover the Expiration picker, its one-column limit, disabled-state explanation, and cell rendering.

Post-Deploy Monitoring & Validation

  • Logs: Search for CleanupTableRowTtl, cleanup-table-row-ttl, and Table row TTL cleanup completed.
  • Healthy: Enabled environments run cleanup every fifteen minutes; batches, deleted, and limitReached stay consistent with table activity; no row is removed before its stored epoch.
  • Failure: Repeated cleanup errors, persistent limitReached, excessive database time, or any early deletion report.
  • Mitigation: Disable table-row-ttl/TABLE_ROW_TTL to stop new TTL use and cleanup dispatch, then roll back if necessary.
  • Window and owner: Monitor for 24 hours after enablement; owner is the Tables feature owner.

Compound Engineering

@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 29, 2026 9:41pm

Request Review

@j15z j15z changed the title feat/table row ttl feat(tables): add automatic row expiration with TTL Aug 25, 2026
@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a TTL table-column type backed by Unix epoch seconds, including validation, timezone-aware editing, import and conversion support, and documentation.

  • Adds scheduled, serialized background cleanup of expired table rows.
  • Enforces one TTL column per table across schema validation and table mutation flows.
  • Adds TTL rendering, editing, picker availability, tests, deployment schedules, and generated tool metadata.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking cleanup-fairness issue that can leave later tables behind under sustained high-volume expiration workloads.

The TTL representation and mutation paths are internally consistent, but the cleanup worker's fixed ordering and shared batch budget allow an early table to repeatedly monopolize scheduled cleanup capacity.

Files Needing Attention: apps/sim/background/cleanup-table-row-ttl.ts

Important Files Changed

Filename Overview
apps/sim/background/cleanup-table-row-ttl.ts Implements locked, batched TTL deletion; the global budget and fixed table ordering can create unbounded lag under sustained backlog.
apps/sim/lib/table/column-types/ttl.ts Defines TTL epoch-second coercion, validation, conversion, and timezone-aware formatting.
apps/sim/lib/table/columns/service.ts Applies source-aware value conversion and validates the resulting schema before retyping columns.
apps/sim/lib/table/schema-invariants.ts Extends shared schema invariants with registry-declared per-table column limits.
apps/sim/app/api/cron/cleanup-table-row-ttl/route.ts Adds an authenticated cron endpoint that dispatches one deduplicated, serialized cleanup job.
apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/table-grid/cells/inline-editors.tsx Normalizes date-editor saves through timezone-aware column coercion so TTL cells persist epoch seconds.
helm/sim/values.yaml Enables the TTL cleanup endpoint on a five-minute non-overlapping cron schedule.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Cron[Five-minute cron] --> Route[Authenticated cleanup route]
  Route --> Queue[Serialized cleanup job]
  Queue --> List[List tables with expired TTL rows]
  List --> Lock[Acquire table advisory lock]
  Lock --> Batch[Delete up to 500 expired rows]
  Batch -->|More rows and budget remains| Lock
  Batch -->|Table complete| Signal[Signal table rows changed]
  Signal -->|More tables and budget remains| Lock
Loading

Reviews (1): Last reviewed commit: "feat(tables): add row TTL expiration" | Re-trigger Greptile

Comment thread apps/sim/background/cleanup-table-row-ttl.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread apps/sim/lib/table/column-types/ttl.ts Outdated
@j15z
j15z force-pushed the feat/table-row-ttl branch from 9a22284 to 0240ade Compare August 26, 2026 18:21
@j15z
j15z requested a review from a team as a code owner August 26, 2026 18:21
@j15z
j15z changed the base branch from main to staging August 26, 2026 18:22
@j15z
j15z force-pushed the feat/table-row-ttl branch from 0240ade to 93d65ea Compare August 26, 2026 19:53
@j15z
j15z changed the base branch from staging to refactor/table-column-type-extension-points August 26, 2026 19:53
@j15z
j15z force-pushed the feat/table-row-ttl branch from b604ffa to f00f927 Compare August 26, 2026 21:32
@j15z
j15z force-pushed the feat/table-row-ttl branch from f00f927 to dc9903c Compare August 26, 2026 21:52
@j15z
j15z force-pushed the feat/table-row-ttl branch from dc9903c to c99a841 Compare August 26, 2026 22:14
@j15z
j15z force-pushed the feat/table-row-ttl branch from 2763e53 to c995a4a Compare August 28, 2026 02:23
@j15z
j15z force-pushed the feat/table-row-ttl branch from 5dda494 to fddd521 Compare August 29, 2026 21:38
@j15z
j15z merged commit 7353b7c into staging Aug 29, 2026
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant