diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index e6bd352a..6d0ea4ed 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,88 +1,61 @@ -# Knowledge flush — 7 insight(s) +# Knowledge flush — 1 insight(s) -7 queued rows claimed: 7 general insights, 4 new pages, 3 merged into existing pages, 0 dropped. One candidate claim was refuted during verification and corrected (see dad0a142c377a6f1). +Headless autoflush run, 2026-11-06 (claims by run `20261106-124649-3684`). Claimed 3 candidates: 1 ingested as a new page, 2 released back to `pending` as pending duplicates of open PRs. -This flush reuses the content of an earlier local commit (29a219b, same base `50a3af6`) that never reached GitHub, because the contributor has read-only access to this repo. Every page was reviewed again and four were corrected. The PR comes from the fork `jocenunes/dev-loop` for the same reason. +| Candidate | Domain | Disposition | +|-----------|--------|-------------| +| `873d73b347fd363a` — group near-duplicate templated examples before CV; threshold from out-of-fold predictions; rejection rate on a held-out set | testing hint → backend/ml | **ingested** — new page `backend-common-ml-grouped-cross-validation-for-templated-examples` (verified) | +| `b5863e2d758c98fb` — Claude Code behind a gateway still sends bootstrap/penguin_mode requests while a claude.ai login remains | platforms | released — pending duplicate of open #328 / #298 | +| `fc76dc4dfa08ecde` — Docker-gated schema validator: skip vs fail, known-bad document | testing | released — pending duplicate of open #300 / #317 | ## Verified best-practice -**9f5c3a177b7d59b1: a markdown code span copied into a gate `CHECK:` line runs as command substitution.** -- Sources: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html (§2.6.3), https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html -- How verified: end-to-end on main `50a3af6`. `plan-gate.sh emit A` kept the backticks in `CHECK:`, and `gate-check.sh --run` recorded `baseline-tests-ran` UNMET with `exit=127`; the bare bullet recorded MET with `exit=0 matched: GATE_OK`. A code span printing `touch ` created the file, so the output really is executed. -- Confidence: verified. -- Owner note: `plan-gate.sh` lines 85 and 369 could strip the backticks. That is a skill change, so it is out of scope here. - -**dad0a142c377a6f1: non-ASCII object keys fail in Convex at runtime.** -- Sources: https://docs.convex.dev/database/types, https://docs.convex.dev/production/state/limits, convex 1.46.0 and 1.47.0 `dist/esm/values/value.js` (`validateObjectField`). -- How verified: `convexToJson` threw for `16–30`, `preço` and an emoji, and passed `16-30`. Through `convexTest`, on convex 1.46.0 with convex-test 0.0.60 and on 1.47.0 with 0.0.62, `t.query` and `t.mutation` threw the same error, including for a nested key. -- Correction: the candidate said `convex-test` passes. That is false for a test that calls the registered function, so the page now says only helper-level tests miss it. -- Confidence: verified. - -**064e6c85d3711615: parallel children patching one parent document exhaust write-conflict retries, and `convex-test` cannot show it.** -- Sources: https://docs.convex.dev/error, https://docs.convex.dev/testing/convex-test, convex-test 0.0.60 `dist/index.js` (`TransactionManager`), @convex-dev/workpool README and 0.4.13 `dist/component/complete.js`. -- How verified: docs give the error text and the data-model remedy. The convex-test source serializes top-level transactions, and a local run of 34 parallel read-and-patch mutations gave 0 rejections and a final count of 34. The docs page does not discuss concurrency, so that claim rests on source and reproduction. The failure on a real deployment is a field report (34 children, 281 mock tests green); it was not reproduced here. -- Correction: the earlier draft advised against throttling, but the Workpool README documents `maxParallelism: 1` as a fix, so it is now a table row. `onComplete` is inline only when `runOnCompleteInline` is set, and the page now says so. -- Confidence: verified. - -**fb8f286a0d295bd3: an `apiSpec` timeout from `npx convex run` hides "Could not find function".** -- Sources: convex 1.46.0 `dist/esm/cli/lib/run.js` lines 52-56. -- How verified: source read: within `convex run`, `apiSpec` is queried only inside the `Could not find function` branch. The 1 s timeout is a field observation; no string for it exists in the npm package, so it comes from the server. -- Confidence: verified for the mechanism. - -**ef1f9fc35666da7f: a target guard must resolve the environment in the CLI's own order.** -- Sources: https://docs.convex.dev/cli/reference/deploy, https://docs.convex.dev/production/project-configuration, convex 1.46.0 `dist/esm/cli/lib/deploymentSelection.js` lines 84-85 and `command.js`, `npx convex deploy --help`. -- How verified: source read plus a local dotenv 17.3.1 run: the same two `config` calls gave the file's value when unset and kept `dev:exported` when exported. `--help` confirms the production default and the `--dry-run` wording. -- Correction: added the flags that override the environment (`--prod`, `--deployment`, `--deployment-name`, `--preview-name`, `--url`, `--admin-key`), which the candidate and the earlier draft omitted. -- Confidence: verified. - -**9e1f0d72258b768a: prove "identical to main" by running the base commit's code against the committed snapshots.** -- Sources: https://git-scm.com/docs/git-archive, https://jestjs.io/docs/cli (`--ci`), https://vitest.dev/guide/snapshot. -- How verified: the tools' behaviour is documented, and Vitest was added beside Jest. The method itself comes from one field review (3 of 3 green, checksums unchanged). -- Confidence: field-tested for the method. - -**b68aa8912bbe777d: three Templater toggles live in device local storage, not `data.json`.** -- Sources: https://raw.githubusercontent.com/SilentVoid13/Templater/master/src/settings/LocalSettings.ts, https://raw.githubusercontent.com/SilentVoid13/Templater/master/src/settings/Settings.ts, https://raw.githubusercontent.com/SilentVoid13/Templater/master/src/main.ts, https://raw.githubusercontent.com/obsidianmd/obsidian-api/master/obsidian.d.ts -- How verified: read current source on master, which matches the installed 2.26.0. `test/specs/migration.e2e.ts` asserts these keys are not migrated. -- Correction: added the reset notice for old `data.json` values, and the vault scope of `App.loadLocalStorage`. -- Confidence: verified. - -All 13 cited URLs returned HTTP 200 on 2026-11-04. +**`873d73b347fd363a` — confidence: verified** (the method is backed by official docs; the specific cutoff and IDF setting are field measurements and the page labels them that way). + +- Claim: when intent-classifier examples come from templates or paraphrases, split CV by near-duplicate groups (`GroupKFold` / `StratifiedGroupKFold`), choose the rejection threshold from out-of-fold predictions only, and measure the rejection rate on a separate out-of-scope set. +- Sources checked (WebSearch; this run has no WebFetch, so pages were confirmed through search summaries, not fetched): + - https://scikit-learn.org/stable/modules/cross_validation.html — §3.1.2.4 "Cross-validation iterators for grouped data": "The i.i.d. assumption is broken if the underlying generative process yields groups of dependent samples"; samples in the validation fold must come from groups not in the paired training fold; `GroupKFold`, `StratifiedGroupKFold`, `GroupShuffleSplit`, `LeaveOneGroupOut`. + - https://scikit-learn.org/stable/modules/classification_threshold.html — §3.3: `TunedThresholdClassifierCV` tunes the decision threshold with internal cross-validation; `cv="prefit"` only with a new validation set. + - https://aclanthology.org/D19-1131/ — Larson et al., EMNLP 2019 (CLINC150): the out-of-scope threshold is chosen on validation data; out-of-scope recall is reported apart from in-scope accuracy. + - Field evidence from the source session: 123 weather templates formed 69 groups; F1 went from 0.854 to 0.828 (the threshold-selection change landed in the same run, and the page says so); one known pair scored 0.66 with IDF and 0.77 without. +- Changed from the candidate: the candidate stated "no IDF, cosine ≥ 0.75" as the rule. The page gives the n-gram/single-link method as one working recipe, adds "use the template id as the group id when you have it", and turns the IDF choice into a measurement step (score known copy and non-copy pairs, then choose). A single cutoff from one dataset is not general. Added `StratifiedGroupKFold` for intents with few groups and an edge row for single-link chaining. +- Not verified externally: that IDF lowers similarity between near-copies in general. This rests on the session's measured pair, and the page cites it as a field measurement. + +**`b5863e2d758c98fb`, `fc76dc4dfa08ecde`** — released before verification (see Open-PR check). ## Existing-layer check -Pages read: infrastructure-agent-orchestration-gate-evidence-exit-code-class, testing-quality-key-order-in-serialized-goldens, testing-quality-value-preserving-refactor-assertions, testing-quality-stale-artifact-baselines, debugging-signals-reading-error-messages, databases-transactions-optimistic-vs-pessimistic-locking +Pages read: backend-common-ml-benchmark-relative-signal-classification, backend-common-ml-mape-aligned-point-prediction, backend-common-llm-korean-summary-line-dedupe-by-bigram-overlap -- `wiki_search` ran on all 7 triggers. Top hits considered: 9f5c → `qa-process-completion-claims`, `platforms-shells-heredoc-body-expansion-with-backtick-prose`, `infrastructure-agent-orchestration-gate-evidence-exit-code-class`, `platforms-shells-command-text-inspected-before-execution`. 9e1f → `testing-quality-key-order-in-serialized-goldens`, `testing-quality-behavior-not-implementation`. The Convex, guard and Templater triggers returned only unrelated pages (best scores about 0.75). -- `grep -i convex` and `grep -i 'obsidian|templater'` over `wiki/` returned nothing on main. -- Merged: 9f5c into `gate-evidence-exit-code-class` (new step, edge rows, sources); fb8f into `reading-error-messages` (edge row, source); 9e1f into `stale-artifact-baselines` (edge row, sources). -- Created: the four pages listed under Routing decision. -- Conflicts: none flagged. The docs/code gap on ASCII field names is recorded as an edge row. -- `related:` links added both ways: `backend-node-boundaries-runtime-validation`, `databases-transactions-optimistic-vs-pessimistic-locking`, `infrastructure-config-environment-config`, `backend-node-runtime-env-files-with-process-loadenvfile`. -- Checks: `node scripts/wiki-structure-checks.js wiki` reports 472 pages, 0 findings; `wiki-lint-prohibitions.js` reports 0 violations. +- Indexes read: `INDEX.md`, `wiki/backend/index.md` (full). +- `wiki_search` top hits considered for the candidate's trigger (3 queries: English trigger, ML-leakage phrasing, the Korean trigger sentence): `platforms-tools-agent-permission-classifier-denials`, `testing-quality-spec-artifact-checks`, `backend-common-ml-benchmark-relative-signal-classification`, `backend-common-llm-korean-summary-line-dedupe-by-bigram-overlap`, `testing-strategy-test-level-choice`. No hit covers model-evaluation leakage, grouped CV or out-of-scope thresholds. The "classifier" hits refer to the permission classifier, which is unrelated. +- `backend/common/ml` holds two model pages (MAPE point prediction, benchmark-relative signals). Neither covers evaluation splits, so this is a **new page** in that category rather than a merge. +- `backend-common-llm-korean-summary-line-dedupe-by-bigram-overlap` uses character-bigram overlap against a threshold for Korean text. It is the closest technique but has a different trigger (removing duplicate summary lines). It is linked, not merged. +- Related links: the new page links to the Korean dedupe page and the MAPE page. Back-links were added to both pages' `related:`. +- Conflicts: none. +- `wiki/backend/index.md`: one row added to the `ml` table. `log.md`: one ingest entry appended. +- Not edited: the backend row in root `INDEX.md` and the `common` row at the top of `wiki/backend/index.md`. Open PR #298 edits `INDEX.md`, and #329/#315/#298 edit `wiki/backend/index.md`, so changing those summary lines would likely conflict. The `ml` table row is enough to route there. Reviewer: add "grouped CV of templated classifier data" to the INDEX.md backend line after those PRs land if you want it in root routing. ## Open-PR check -Listed 22 open `knowledge/*` heads: #293, #298-#301, #304-#310, #315, #317-#319, #324, #328-#332. Diffed the 19 same-repo heads through `origin/`, and the 3 fork heads (#298, #299, #300) through `pull//head`, for Convex, Templater, Obsidian, `git archive`, workpool, `.env`, backtick, snapshot and command substitution. A title and body search for Convex or Templater returned nothing. +22 open `knowledge/*` PRs: #332, #331, #330, #329, #328, #324, #319, #318, #317, #315, #310, #309, #308, #307, #306, #305, #304, #301, #300, #299, #298, #293. This headless run cannot fetch or diff branch content, so overlap was judged from each PR's title and changed-file list (`gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/" --json number,title,files`). -- All 7 candidates: **new**. Keyword hits in #299, #309, #317 and #329 are unrelated uses of "snapshot" or "command substitution". -- Adjacent, not duplicate: #301 covers UTF-8 byte bounds for client JSON keys, which is a different trigger from Convex field names. #300 edits `environment-config.md`, which this PR touches only in `related:`, so a trivial merge conflict is possible. +| Candidate | Overlapping open PR | Verdict | +|-----------|---------------------|---------| +| `873d73b347fd363a` grouped CV for templated examples | None. No open PR touches `backend/common/ml/`, model evaluation, or classifier thresholds. #315/#310/#301/#293/#317/#318/#319 touch `wiki/testing/index.md` rows on other triggers. #329 edits `wiki/backend/index.md` in the `api-design` section, not `ml` | **new** | +| `b5863e2d758c98fb` gateway + leftover claude.ai login | #328 edits `platforms/processes/spawning-an-agent-cli-from-a-server.md` with `ANTHROPIC_BASE_URL`/`ANTHROPIC_AUTH_TOKEN` gateway rows. #298 adds `backend/common/llm/gateway-model-alias-resolution.md`. Same trigger family (Claude Code behind a gateway). This matches the verdicts of runs #329, #330, #331 and #332, and both PRs are still open | **pending duplicate → claim released** | +| `fc76dc4dfa08ecde` Docker-gated schema validator: skip vs fail | #300 edits `testing/quality/checks-that-cannot-pass.md` and `tests-that-cannot-fail.md`. #317 edits `checks-that-cannot-pass.md`. A validator that is skipped or always says "valid" is their theme. This run read the row (queue file located through `~/.claude/history.jsonl`). The verdict is the same as runs #318, #319, #324 and #328, and both PRs are still open | **pending duplicate → claim released** | ## Routing decision -| Insight | Target | Action | -|---------|--------|--------| -| 9f5c3a177b7d59b1 | infrastructure/agent-orchestration/gate-evidence-exit-code-class | merge | -| dad0a142c377a6f1 | backend/node/convex/ascii-keys-in-returned-objects | new page | -| 064e6c85d3711615 | backend/node/convex/parallel-children-writing-one-parent-document | new page | -| fb8f286a0d295bd3 | debugging/signals/reading-error-messages | merge | -| ef1f9fc35666da7f | infrastructure/deploy/effective-target-guard-for-a-deploy-cli | new page | -| 9e1f0d72258b768a | testing/quality/stale-artifact-baselines | merge | -| b68aa8912bbe777d | platforms/tools/templater-device-local-settings | new page | - -New category `backend/node/convex`: no existing node category (runtime, async, boundaries, language) fits a hosted backend's value rules and transaction model, and two pages share it. +- `873d73b347fd363a` → `backend` / `common/ml` → new page `wiki/backend/common/ml/grouped-cross-validation-for-templated-examples.md`. The candidate's domain hint was `testing`, but the testing domain covers automated tests of code (levels, fixtures, mocks, flaky tests). This lesson is about evaluating a trained model's score, which the existing `backend/common/ml` category (model output calibration, signal grading) owns. The directive is general: any classifier trained on templated or paraphrased text. The source project appears only in a Sources line. No new category. ## Local-layer candidates none -🤖 Generated with [Claude Code](https://claude.com/claude-code) +## Queue bookkeeping + +- Retired in place (`status: retired`, `retiredBy`, `retiredAt`, `outcome`), the same way earlier headless runs retired rows: `873d73b347fd363a`. +- Released back to pending with `queue-claim.js release`: `b5863e2d758c98fb`, `fc76dc4dfa08ecde`. +- Reviewer note: `b5863e2d758c98fb` and `fc76dc4dfa08ecde` have now been released by six or more consecutive runs while #298/#300/#317/#328 stay open. Merging or closing those PRs, or one manual `/dev-loop:knowledge-flush` (which can diff them), would clear them. diff --git a/log.md b/log.md index 109cac0a..1062f01c 100644 --- a/log.md +++ b/log.md @@ -435,3 +435,4 @@ Append-only. Format: `## [YYYY-MM-DD]