[leemour]

Task

notion-cli plan C10, owner's ruling NEED-2639 A: lift the per-account catalog ceiling and make the catalog import and the local document listing work in portions, so a sync goes past any total. A single call may stay bounded.

Why

A Notion workspace synced whole (the default, NEED-2636) passes 10,000 pages. cli-core refused such an account: catalog.list above 10,000 entries / 32 MB, catalog.reconcile and the ingestion import above 10,000 ids, and the ingestion listing stopped at 10,000. Drive has the same limit.

What

• src/store/sqlite/catalog.ts: the per-account guard in list is gone (it also counted the whole account on every page); reconcile takes any number of ids.
• src/services/documents/ingestion.ts: the import takes any number of files, upserting and reporting observed 200 at a time. The whole-account note cache is gone: it read every document (with text) in pages of 500 that each re-read all rows before them. Each save and removal now looks up its one document with the new notes.fileNote(folderId, path) (src/store/sqlite/file-note.ts, indexed by the documents key, or the files row of a local folder).
• Per-call bounds stay: 200 entries per catalog write, 1000 per list page, 1 MB per entry.
• docs/dev/DOCUMENTS.md: the <id>.md external id contract that the shared catalog commands rely on (lane L2 asked for it to be either documented or removed; documented here), and the bounds.
• feat!: an own NotesStore implementation must add fileNote; the catalog.scope-limit error is no longer raised (the reason stays registered).

Measured

cli-testing#375, performance/documents/catalog-scale: before, over 10,000 entries cannot be imported. After, 100,000 entries import in 8.6 s with a 489 MB peak; 200 saves take 0.3 s at any size (2.4 s at 9,800 before, 37 s at 100,000 with the old listing); a list page takes 60 ms. The import runs on every sync, also when nothing changed (about 7 s at 100,000): the next thing to speed up, not a blocker.

Testing

• New src/services/documents/ingestion-scale.test.ts: 10,050 entries import, list past 10,000, save twice to the same document, retire one by re-import, remove.
• src/store/catalog.test.ts: the reconcile-over-10,000 case now expects success.
• Locally: typecheck; src/services/documents src/documents src/store 2116 passed before the last test update, then the touched files 112 passed; biome, docs checks green.
• Not stacked: independent of feat(documents): any documents provider gets documents accounts, not only Drive #2938 (C1) and feat(documents): a synced node can be a folder and a document at once #2941 (C2).

🤖 Generated with Claude Code