Skip to content

Threads: entries cite KB articles via kb_refs (#978) - #982

Merged
mkreyman merged 2 commits into
masterfrom
feature/thread-entry-kb-refs
Oct 15, 2026
Merged

mkreyman merged 2 commits into
masterfrom
feature/thread-entry-kb-refs

Conversation

@mkreyman

@mkreyman mkreyman commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Closes #978.

A message or checkpoint thread entry can now name the knowledge articles it acted on, as data, so a review reads which KB decision each checkpoint relied on instead of scraping ids out of a body.

What changed

  • Schema: thread_entries.kb_refs uuid[] NOT NULL DEFAULT '{}' (catalog-only on PG 11+, no rewrite). Two CHECKs restate the shape rules so a write that bypasses the context cannot store refs on another kind or past the bound: thread_entries_kb_refs_bound (cardinality <= 20) and thread_entries_kb_refs_kinds (empty, or kind is message/checkpoint).
  • Threads.Entry: kb_refs is cast canonical (lowercased UUIDs), de-duplicated keeping first-cited order, and bounded on DISTINCT ids, so repeats do not use up the bound. A kind other than message/checkpoint carrying refs is a changeset error.
  • Threads.record_entry/4 and record_checkpoint/3: every id is checked against the articles the writing key can read (Knowledge.article_refs/3: the tenant's articles under the KB visibility scope, plus published system canonicals) BEFORE the story lock or any write. One unknown id refuses the whole entry with 422 unknown_kb_refs naming it. Ids only are stored, never bodies. The audit-chain payload carries kb_refs when non-empty, so the references are chained with the entry.
  • Idempotency: kb_refs joins the replayed fields. A message resend with other refs is 409 idempotency_key_reused; a checkpoint resend that SENDS refs must send the recorded ones (409 checkpoint_conflict), and one that omits them replays.
  • Read side: thread_get returns kb_refs on each entry. The review payload (GET .../review) adds kb_articles beside each entry: {id, readable: true, title, title_untrusted: true, category, status} for an article the reading key can see, {id, readable: false} for one it cannot.
  • OpenAPI: entry and checkpoint request bodies document kb_refs from one @kb_refs_schema attribute; the 409/422 descriptions name the new codes.
  • MCP 2.116.0: thread_entry and thread_checkpoint accept kb_refs (array of article ids, client-side shape check); thread_get and thread_review_get descriptions say what comes back. README rows and CHANGELOG updated.

Defaults Mark may overrule

  • Bound of 20 distinct articles per entry (repeats are dropped before counting) (Entry.max_kb_refs/0, restated in the migration). Raising it is a new CHECK migration.
  • Citable set = what get_article/3 resolves: the tenant's own articles the key can read, plus published system canonicals.
  • unknown_kb_refs is one answer for missing and unreadable, so the endpoint is not an existence oracle for another agent's private memory.
  • A reader who cannot see a cited article gets readable: false with no title, even though the writer could see it.

Filed separately (label deferred-finding)

Tests

  • test/loopctl/threads/kb_refs_test.exs: canonicalisation and order, unknown id, tenant isolation, visibility, bound/shape refusals, de-duplicated bound, idempotent resend, audit payload, checkpoint path and replay, changeset kind rule, both DB constraints.
  • Controller tests for entry/checkpoint 201/422 and the review payload's kb_articles.
  • mcp-server/test/threads_tool.test.js for the tool wiring and client-side shape check.

Review round (/code-review high)

Fixed:

  1. A resend was re-checked against current visibility before replay (threads.ex record_entry/record_checkpoint), so an article made private after the first write turned a retry into 422. The verdict is now computed before the lock and applied only to a new write.
  2. A malformed kb_refs on a resend read as the default [] and replayed as the same write. An invalid changeset is now refused before the lock.
  3. :visibility_agent_id defaulted to "unscoped" when omitted. It is now required (nil = every article) whenever refs are named, and by Reviews.payload/4.
  4. [null] and a 16-byte string passed Ecto.UUID's cast. cast_kb_refs now takes only 36-character UUID text.
  5. The OpenAPI maxItems: 20 contradicted the distinct-id bound; spec and MCP text now say 20 DISTINCT.
  6. System canonicals could not be cited or titled. Citing and titling now share one query (Knowledge.article_refs/3 on visible_article_scope), which also removes the duplicate lookup.
  7. The migration's CHECKs validated under the ADD's ACCESS EXCLUSIVE lock. They are now NOT VALID plus separate VALIDATE with the DDL transaction disabled, as 20260723120000 does.

Not fixed, false positive: "thread_get returning a cited private article's id is an existence oracle." An oracle answers a prober-chosen input. Here the reader cannot choose the id; it sees only ids a writer deliberately put in a shared thread (the same ids the audit chain payload carries). It gets readable: false with no title, category or body. The write-side single answer for missing and unreadable articles still stands, and that is the only place a caller can probe arbitrary ids.

A message or checkpoint entry now carries kb_refs, the knowledge article
ids it acted on, so a review reads which KB decision each checkpoint
relied on as data instead of scraping ids out of a body.

- thread_entries.kb_refs uuid[] NOT NULL DEFAULT '{}', with CHECKs for
  the bound (20) and the kinds (message, checkpoint).
- Each id must be an article of the tenant the writing key can read
  (agent keys are visibility-scoped); anything else refuses the whole
  write 422 unknown_kb_refs, naming the id. Checked before the story
  lock, since it reads on AdminRepo.
- kb_refs is part of the replayed write and rides in the entry's
  audit-chain event when non-empty.
- thread_get returns kb_refs per entry; the review payload renders each
  cited article's title, category and status beside the entry, or
  readable: false for one the reading key cannot see.
- MCP 2.116.0: thread_entry and thread_checkpoint accept kb_refs.

Runner-socket writes (#979) and recording a citation as an article use
(#980) are filed separately.
@mkreyman

Copy link
Copy Markdown
Owner Author

Review round 1 (code) counted at e30e1181af6d via Skill code-review. The review-round counter's merge gate reads this marker on any machine.

- A resend is answered from its row: the citable-set read still runs
  before the story lock, but its verdict binds only a new write, so an
  article made private or deleted after the first write cannot turn a
  retry into a refusal of an entry or checkpoint that already landed.
- An invalid entry changeset is refused before the lock. A kb_refs that
  failed its cast read as the default [], so a malformed resend matched
  a row citing nothing and was acknowledged.
- :visibility_agent_id is required (nil = every article) whenever
  kb_refs names any, and by Reviews.payload/4: omitting it fails instead
  of reading unscoped.
- cast_kb_refs takes only 36-character UUID text: [nil] and a 16-byte
  string no longer pass as ids the caller never sent.
- Published system canonicals are citable and titled. Citing and the
  review's titles now use one query, Knowledge.article_refs/3, built on
  get_article/3's visible_article_scope.
- OpenAPI and MCP describe the bound as 20 DISTINCT ids (repeats are
  dropped first) instead of a raw maxItems the server does not enforce.
- The migration adds both CHECKs NOT VALID and validates them in their
  own statements, DDL transaction disabled, so the scans hold only
  SHARE UPDATE EXCLUSIVE and thread writes continue.
@mkreyman
mkreyman enabled auto-merge (squash) October 15, 2026 14:59
@mkreyman
mkreyman merged commit f436944 into master Oct 15, 2026
16 checks passed
@mkreyman
mkreyman deleted the feature/thread-entry-kb-refs branch October 15, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Thread entries: kb_refs so a review sees the KB decisions behind each checkpoint

1 participant