Skip to content

fix(knowledge): address release review follow-ups for ACL pages, projection fill, and sync retries - #8202

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/knowledge-release-review-followups
Sep 23, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/knowledge-release-review-followups

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Scope the members-mode observed ACL to members of the document's own connector, so an observation from another connector can never grant a document
  • Advance a member's feed cursor and watermark only after the ACLs its listing decides are written; a replayed feed removal rematerialises every document it names
  • Plan connector-lease ACL pages from an unlocked read and lock only the page that will be written, re-reading and trimming under the lock
  • Lock projection fill documents FOR KEY SHARE SKIP LOCKED so a concurrent delete can no longer fail the pass with FK 23503
  • Report an unfinished projection fill as remaining work so the next pass is requested immediately
  • Record database_failure_class on failed sync-log rows (expand-only migration 0379) and count only database failures toward the database retry streak and alert
  • Heartbeat the lease between revokeDocumentAcls transactions
  • Extract isTriggerAvailable to lib/core/config/trigger-availability and use it for every Trigger.dev dispatch decision
  • Rewrite the processing lock-scope deletion test to commit the deletion from a second connection during the embedding writes

Type of Change

  • Bug fix

Testing

  • New Postgres tests: re-owned document ACL, page lock past a held document and concurrent chunk growth, fill under a concurrent delete, deletion committed during embedding writes
  • New unit tests: checkpoint ordering and feed replay, database failure streak, fill remaining, revoke heartbeat, trigger availability
  • Each new test fails against the pre-fix code (mutation-checked)
  • Knowledge Postgres CI steps, lint, type-check, check:audits, check:api-validation, check:migrations, docs-manifest:check

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…ection fill, and sync retries

- Scope the observed-ACL aggregate to members of the document's own connector
- Advance a member's listing checkpoint only after the ACLs it decides are written,
  and rematerialise every document a replayed feed removal names
- Plan ACL pages from an unlocked read and lock only the page that will be written
- Lock fill documents FOR KEY SHARE SKIP LOCKED so a concurrent delete cannot fail
  the mark's foreign key
- Report an unfinished projection fill as remaining work
- Record the database failure class on failed sync-log rows and count only those
  toward the database retry streak (expand-only migration 0379)
- Heartbeat the lease between revokeDocumentAcls transactions
- Extract isTriggerAvailable to lib/core/config/trigger-availability and use it
  for every dispatch decision
- Exercise a concurrently committed deletion in the processing lock-scope test
@vercel

vercel Bot commented Sep 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 23, 2026 4:21pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The changes since the previous review appear safe to merge, with no new actionable defects identified.

Summary

This PR hardens knowledge synchronization and projection processing around concurrency, ACL correctness, checkpoint ordering, lease maintenance, and database-specific retry accounting.

  • Restricts observed ACL subjects to active members of the document’s current connector.
  • Materializes listing-driven ACL changes before advancing member cursors and watermarks.
  • Reduces ACL page lock scope and heartbeats leases between page transactions.
  • Protects projection fill against concurrent document deletion and requests another pass after an unfinished fill.
  • Records validated database failure classes and uses them to calculate retry streaks.
  • Centralizes Trigger.dev availability decisions and expands concurrency-focused coverage.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Member listing or change-feed page] --> B[Write observations and removals]
  B --> C[Rematerialize affected document ACLs]
  C -->|success| D[Advance member cursor and watermark]
  C -->|failure| E[Leave checkpoint unchanged]
  E --> A

  F[Projection fill scans unfilled rows] --> G[Choose bounded documents]
  G --> H[Acquire key-share locks with skip-locked]
  H --> I[Insert dirty projection marks]
  I --> J{Budget or rows remain?}
  J -->|Yes| K[Report remaining work]
  J -->|No| L[Complete pass]
Loading

Reviews (2) · Last reviewed commit: "fix(knowledge): constrain the recorded d..."

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 41 files

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Fix all with cubic | Re-trigger cubic

Comment thread packages/db/schema.ts
@greptile-apps

greptile-apps Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

  • P2 Skipped rows advance cursor packages/db/knowledge-projection.ts:514

    A document skipped by FOR KEY SHARE SKIP LOCKED still contributes to last_id. An ordinary concurrent document update can therefore make the fill advance past projection rows it never marked. Updates that do not change connector_id or acl do not independently enqueue projection work, so the pass can return cursor: null and remaining: false while those rows remain unfilled. The Trigger task then skips the immediate follow-up pass, delaying convergence until the periodic sweep. Compute the cursor from successfully locked documents, or preserve remaining work when a chosen row is skipped.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 41 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit aba70b2 into staging Sep 23, 2026
25 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/knowledge-release-review-followups branch September 23, 2026 16:24

This branch was successfully deployed

1 active deployment
Preview 99393a6b Deployed Sep 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant