Skip to content

fix(storage): index the columns behind ON DELETE SET NULL - #34

Merged
denfry merged 1 commit into
mainfrom
fix/index-foreign-key-columns
Nov 2, 2026
Merged

denfry merged 1 commit into
mainfrom
fix/index-foreign-key-columns

Conversation

@denfry

@denfry denfry commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Problem

symbols.parent_id and chunks.symbol_id reference symbols(id) with ON DELETE SET NULL but have no index. With foreign_keys = ON, every deleted symbol scans both tables, so removing files costs O(deleted symbols x table rows) and runs into hours on a large index.

Solution

Add idx_symbols_parent and idx_chunks_symbol to schema.sql and mirror them in docs/SCHEMA.md.

Verification

  • Synthetic index (3000 files, 18k symbols, 45k chunks), deleting 100 files: 4.68 s before, 0.03 s after. Creating the indexes takes 0.07 s, and foreign_key_check stays clean.
  • Two new tests, both failing before the fix: every foreign-key column must lead an index, and an existing index gains the new ones on open.
  • Full suite and ruff pass.

Compatibility

_apply_schema() runs on every open with IF NOT EXISTS, so existing indexes pick these up on their next open. No schema version bump, no rebuild.

Fixes #33

Every deleted symbol looked up its children in chunks and symbols with a full table scan, so removing files from a large index took hours. Existing indexes gain both indexes on their next open.

Fixes #33
@denfry
denfry merged commit d55a6f0 into main Nov 2, 2026
11 checks passed
@denfry
denfry deleted the fix/index-foreign-key-columns branch November 2, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deleting files takes hours on large indexes: chunks.symbol_id and symbols.parent_id have no index

1 participant