Skip to content

feat(organize): themes across earlier uncategorized files, re-rank on folder edits, remember description changes - #117

Open
iLevyTate wants to merge 2 commits into
mainfrom
feat/uncategorized-history-folder-learning
Open

iLevyTate wants to merge 2 commits into
mainfrom
feat/uncategorized-history-folder-learning

Conversation

@iLevyTate

Copy link
Copy Markdown
Owner

Summary

Stacked on #116 (this branch contains its commit until that merges; only the last commit is new).

  • Theme suggestions use every analyzed uncategorized file. The "Create folder for " offer in Organize now also counts files analyzed in earlier runs that are still uncategorized (new analysis-history:get-uncategorized channel, lean payload, newest 500). The theme must include at least one file on screen, and earlier files stop counting once a folder matches them.
  • Folder edits re-rank without a reanalysis. Organize re-ranks pending files when a folder's description or keywords change (before, only id, path or name did). It uses stored vectors, with no LLM call and no confirmation.
  • Description changes are remembered. Editing a folder description drops the keywords the AI derived from the old wording and records one memory per folder ("" is for: ), aimed at that folder, with up to five earlier wordings kept.
  • Settings shows what was learned. The existing Memory & Rules panel (list, add, edit, delete) was never mounted; it is now in Settings > Performance and shows each folder's earlier wordings.

Scope

  • Area(s): organize, smart folders, suggestions/learning, settings, analysis history IPC
  • Risk Level: medium
  • Breaking Changes: none. Behaviour change: editing a folder description clears its AI-derived keywords / semanticTags unless the edit supplies keywords.

Type of Change

  • Bug fix
  • Enhancement
  • New feature
  • Refactor
  • Documentation update
  • Breaking change

Test Plan

Automated

  • npm run format:check (ran prettier on the changed files only)
  • npm run lint (ran eslint on the changed files only)
  • npm test (ran the affected suites only: 34 suites around smart folders, settings, organize suggestions, analysis history, preload; full run left to CI)
  • Additional tests added/updated when behavior changed

Also ran: typecheck:js, generate:channels:check, verify:ipc-handlers, verify:ipc-handler-schemas, verify:ipc-contracts.

Manual QA

  • Manual checklist completed (see docs/TESTING.md)
  • Not applicable (explain below)

Manual QA notes:

  • Not run in the app. Worth checking by hand: analyze one file from a vendor that already has two files in Uncategorized and confirm the offer appears with "(2 analyzed earlier)"; edit a folder description in Setup, return to Organize and confirm the cards re-rank; open Settings > Performance > Memory & Rules and confirm the description change is listed.

Checklist

  • I scoped this PR to one logical change
  • I self-reviewed code and naming for clarity
  • I updated docs/config references where needed
  • I called out follow-up work or known limitations

Known limitations:

  • Creating the suggested folder does not move earlier files that already sit in the Uncategorized folder; they only count toward the suggestion.
  • A folder-description memory nudges ranking by at most 0.15 and only for files whose content resembles the description; the folder's own vector (rebuilt on edit) remains the main signal.
  • An edit made in Setup while Organize is open in another window does not broadcast; Organize picks it up when its folder list reloads.

…es honestly, drop prompt echoes

From a review of 85 real results (33 landed in Uncategorized):

- Archives: the category was hardcoded to 'archive' although the entry
  listing was read, so every zip of .stl parts reached Uncategorized. The
  dominant entry type (at least half the files) now picks the folder via
  getIntelligentCategory, taken only when it lands on one of the user's
  folders.
- Filename fallback matched folder words as substrings ("Work" in
  "network", "code" in "barcode"). Folder words now match whole words,
  plural-tolerant.
- Filename-only results reported 65-75%, above real content analyses.
  createFallbackAnalysis, the Office fallback and an unmatched archive now
  report FILENAME_ONLY_CONFIDENCE (50).
- dropPromptEchoes clears summary/purpose/project/reasoning/type/entity
  when the model copied a schema instruction or a folder description into
  them, and caps confidence at 50. Run over 1,164 stored results it clears
  64 files, all genuine copies, no false positives. Applied in the
  document, image and media paths.
- calibrateConfidence caps at 65 (below auto-organize's 75) when the LLM's
  folder is kept although the embedding confidently prefers another.
- Organize's "Create folder for <theme>" grouped on one field by exact text,
  which for a fallback analysis is the file's own name, so it never fired.
  findRecurringTheme groups on named entities and shared keywords (one
  tally per term), skipping file-name keys and form words; still 3+ files.

checkjs floor 493 -> 494 for the new uncategorizedThemes.js.
… folder edits, remember description changes

- The 'Create folder for <theme>' offer counts files analyzed in earlier runs
  that are still uncategorized (new analysis-history:get-uncategorized channel),
  as long as the theme includes a file on screen.
- Organize re-ranks pending files when a folder's description or keywords
  change, from stored vectors, with no reanalysis.
- Editing a folder description drops keywords derived from the old wording and
  records the change as a memory aimed at that folder, with the earlier
  wordings kept.
- Settings > Performance shows the Memory & Rules panel so learned memories can
  be reviewed, edited or deleted.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant