Skip to content

perf(search): the coverage step of every message search, and word-end patterns - #356

Open
leemour wants to merge 2 commits into
mainfrom
perf/search-coverage-step
Open

leemour wants to merge 2 commits into
mainfrom
perf/search-coverage-step

Conversation

@leemour

@leemour leemour commented Nov 11, 2026

Copy link
Copy Markdown
Member

Why

The 2026-11-11 search audit (PERF-100, PERF-99) found that every cli-core message search spent about 0.48 s at 1M messages on its coverage step, and that a frequent word end such as *ени was refused before any message was checked. The cli-core fix ships in the search-gaps pull request; this page records the measurement that justified it.

What

  • performance/search/coverage-step/README.md: why, setup, results at 100k and 1M before/after, what it showed, what was not measured.
  • bench.mjs and the raw results/.
  • Linked from performance/search/README.md and PROCESS.md.

Headline

1M before 1M after
coverage read 445.5 ms 0.2 ms
договор* 478 ms 31 ms
*ени refused 50 hits, 647 ms

Synthetic stores only.

🤖 Generated with Claude Code

… patterns

cli-core's message search counted and dated every message of every chat for its
coverage on each search: 0.45 s at 1M messages, whatever the query. Read off the
trigger-kept chat counts and each chat's newest message on its time index, the
step takes 0.2 ms and `договор*` falls from 478 ms to 31 ms. A frequent word end
(`*ени`) is checked up to the page and answers instead of being refused.

Synthetic stores of 100k and 1M messages, before (cli-core 0.641.0) and after
(fix/search-gaps), p50 of 7 warm runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
leemour added a commit to WireCatLabs/cli-core that referenced this pull request Nov 12, 2026
…faster coverage (#2871)

## Task

Fix the search gaps of the 2026-11-11 audit that the owner approved
(NEED-2560 answer A, TASK-986), in one PR. The
shape of search results and pagination is unchanged.

## Why

Agents and people wrote queries that found nothing and said nothing
(`status:open`, `invoice^2`, `author:` in messages),
used two names for one concept, or hit a refusal that named no
alternative. Every message search also spent about
0.45 s at 1M messages counting every chat's messages for its coverage.

## What

| Gap | Example | Before | After |
|---|---|---|---|
| BUG-740 unknown field | `status:open` | searched as the words, 0 hits,
no error | `query.unknown-field`: `tasks list status:open`, or quote it
|
| BUG-740 notes field in messages | `author:x`, `site:x` | 0 hits, no
error | `query.unsupported-field`: "a notes field — search notes reads
it" |
| BUG-741 boost | `invoice^2` | searched the text "invoice^2", 0 hits |
`query.unsupported-operator`: drop it (`invoice`) or quote it |
| UX-221 aliases | `sender:olga`, `title:Альфа` | sender: unknown;
title: matched nothing in messages | `sender:` is `from:`; `title:` is
the chat title (messages) and the subject (mail); one mapping table in
the guide |
| UX-224 presence | `_exists_:from`, `subject:*` | refused / unsupported
| keeps messages that have a value; other fields refuse with `has:file`,
`has:link`… |
| UX-223 tasks | `tasks list request`, `title:request` |
`query.bare-word` / title prefix only | finds "Review the request" |
| UX-223 contacts | `contacts list phone:+34`, `email:example.com` |
`+34` misparsed; no email/phone | matched (phone by digits) |
| UX-223 deals | `search all title:Поставка` | words anywhere in the
deal | the deal's or lead's title only |
| UX-222 reasons | `chat:Альфа*` fitting nothing; 2-letter chat search |
bare not_found / validation | `query.no-match` with `chat:*Альфа*`;
`query.short-text` with a hint |
| PERF-99 word end | `*ени` at 1M | refused (candidate rows), no term
named | 50 hits in 647 ms; a page it cannot fill names the term and
suggests `*part*` |
| PERF-100 coverage | `договор*` at 1M | 478 ms | 31 ms (coverage read
445 ms → 0.2 ms) |
| DEBT-326 | `after:` | reason `legacy_date_field` (not registrable) |
`query.legacy-date-field`, registered; the reasons test now also reads
`searchError(...)` |
| DEBT-327 | `bin/verify-query-reference` on JDK 25 | failed on Java
object ids | passes on JDK 21 and 25 |

Not done, and why:

- **Tasks in `search all`**: memo's MCP gates each `search all` lane by
permission and has no tasks gate, so a tasks
lane would show task titles where `tasks.list` is denied. It needs
memo's gate in the same bump.
- **Meeting description, and `title:` in `meetings search`**: needs a
new hit scope or a folded description column
  (a migration). Meeting titles already match in `search all`.

Breaking items are under "Changed — may break callers" in [the changelog
fragment](changes/fix-search-gaps.md).

Measurement: [cli-testing,
performance/search/coverage-step](https://github.com/WireCatLabs/cli-testing/tree/main/performance/search/coverage-step)
(WireCatLabs/cli-testing#356).

## Testing

- Each new or changed test fails on main's sources (25 tests in 14
files) and passes here.
- typecheck, lint, docs:check and the full suite pass locally;
verify-query-reference passes on JDK 21 and 25.
- Every behavior in the table was run against a seeded synthetic store,
before (main's dist) and after.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant