Repository navigation
Conversation
This was referenced Oct 30, 2026
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved retry and write-gating races can leave indexing workers running untracked and allow index writes to overlap collection deletion.
5 open issues
What changed in this PR
This PR isolates code-index parsing in per-workspace worker threads to contain parser crashes and hangs, addressing #23 without blocking the shared extension host.
Changes:
- Adds worker-based parsing with timeouts, tree cleanup, and explicit recovery after fatal failures.
- Adds cancellation and write coordination across scans, file watching, vector storage, and cache updates.
- Adds lifecycle regression tests, worker build support, and translated failure messages.
| File | Description |
|---|---|
src/services/tree-sitter/languageParser.ts |
Supports worker-local parser runtime assets. |
src/services/code-index/vector-store/qdrant-client.ts |
Propagates deletion failures. |
src/services/code-index/vector-store/__tests__/qdrant-client.spec.ts |
Tests deletion error propagation. |
src/services/code-index/service-factory.ts |
Shares workspace parser and write gate. |
src/services/code-index/processors/scanner.ts |
Gates writes and handles cancellation and fatal parsing. |
src/services/code-index/processors/parser.ts |
Adds worker delegation, tree cleanup, and chunking optimizations. |
src/services/code-index/processors/file-watcher.ts |
Integrates worker parsing and guarded replacement writes. |
src/services/code-index/processors/__tests__/scanner.spec.ts |
Tests cancellation, reconciliation, and batch completion. |
src/services/code-index/processors/__tests__/parser.spec.ts |
Tests tree cleanup and chunking performance. |
src/services/code-index/processors/__tests__/file-watcher.spec.ts |
Tests watcher shutdown and replacement lifecycle. |
src/services/code-index/parser-errors.ts |
Defines fatal parser and cancellation errors. |
src/services/code-index/parse-worker.ts |
Adds the parsing worker entry point. |
src/services/code-index/parse-worker-client.ts |
Queues parses and terminates failed or timed-out workers. |
src/services/code-index/orchestrator.ts |
Coordinates fatal shutdown and guarded indexing writes. |
src/services/code-index/manager.ts |
Tracks parser failures and recreates stopped services. |
src/services/code-index/interfaces/file-processor.ts |
Adds cancellation and fatal-error event contracts. |
src/services/code-index/interfaces/embedder.ts |
Adds optional embedding cancellation. |
src/services/code-index/index-write-gate.ts |
Tracks outstanding writes and uncertain file outcomes. |
src/services/code-index/embedders/openai.ts |
Forwards cancellation to embedding requests. |
src/services/code-index/embedders/__tests__/openai.spec.ts |
Tests cancellation without retries. |
src/services/code-index/constants/index.ts |
Adds parser and stop timeout constants. |
src/services/code-index/__tests__/parse-worker-client.spec.ts |
Tests worker ordering, failures, cancellation, and isolation. |
src/services/code-index/__tests__/orchestrator.spec.ts |
Tests fatal shutdown and late-callback suppression. |
src/services/code-index/__tests__/manager.spec.ts |
Tests failure guards and explicit restart. |
src/services/code-index/__tests__/index-write-gate.spec.ts |
Tests cancellation barriers and uncertain writes. |
src/i18n/locales/zh-TW/embeddings.json |
Adds Traditional Chinese parser-failure guidance. |
src/i18n/locales/zh-CN/embeddings.json |
Adds Simplified Chinese parser-failure guidance. |
src/i18n/locales/vi/embeddings.json |
Adds Vietnamese parser-failure guidance. |
src/i18n/locales/tr/embeddings.json |
Adds Turkish parser-failure guidance. |
src/i18n/locales/ru/embeddings.json |
Adds Russian parser-failure guidance. |
src/i18n/locales/pt-BR/embeddings.json |
Adds Brazilian Portuguese parser-failure guidance. |
src/i18n/locales/pl/embeddings.json |
Adds Polish parser-failure guidance. |
src/i18n/locales/nl/embeddings.json |
Adds Dutch parser-failure guidance. |
src/i18n/locales/ko/embeddings.json |
Adds Korean parser-failure guidance. |
src/i18n/locales/ja/embeddings.json |
Adds Japanese parser-failure guidance. |
src/i18n/locales/it/embeddings.json |
Adds Italian parser-failure guidance. |
src/i18n/locales/id/embeddings.json |
Adds Indonesian parser-failure guidance. |
src/i18n/locales/hi/embeddings.json |
Adds Hindi parser-failure guidance. |
src/i18n/locales/fr/embeddings.json |
Adds French parser-failure guidance. |
src/i18n/locales/es/embeddings.json |
Adds Spanish parser-failure guidance. |
src/i18n/locales/en/embeddings.json |
Adds English parser-failure guidance. |
src/i18n/locales/de/embeddings.json |
Adds German parser-failure guidance. |
src/i18n/locales/ca/embeddings.json |
Adds Catalan parser-failure guidance. |
src/esbuild.mjs |
Builds and watches the parser worker bundle. |
knip.json |
Registers the parser worker entry point. |
Review effort: Balanced
Comment on lines
+27
to
+28
| await this.ready(signal) | ||
| throwIfAborted(signal) |
Comment on lines
+248
to
+250
| if (this.parserFailed || this.servicesStopped) { | ||
| this.parserFailed = false | ||
| await this._recreateServices() |
Comment on lines
+91
to
93
| if (signal.aborted) return | ||
| if (summary.batchError) { | ||
| console.error(`[CodeIndexOrchestrator] Batch processing failed:`, summary.batchError) |
Comment on lines
+400
to
+401
| const points = pointsForBatchUpsert.slice(pointOffset, pointOffset + this.batchSegmentThreshold) | ||
| await this.writeGate.run(this.signal, () => this.vectorStore!.upsertPoints(points), replacementPaths) |
| filePath, | ||
| ]) | ||
| } | ||
| await this.writeGate.run(signal!, () => this.cacheManager.deleteHash(filePath), [filePath]) |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #23
What is the problem?
A failed or stuck Tree-sitter parse can disrupt indexing and editor responsiveness.
How does this PR solve the problem?
Indexing parses in a terminable per-workspace worker thread. Crashed or timed-out parsers are never reused. Indexing fail-stops to Error with one localized message and restarts only on explicit retry. A per-workspace write gate prevents stale attempts from writing to the vector store or cache. Tree cleanup (tree.delete()) is adopted from upstream draft PR RooCodeInc#10390.
Human review: the user approved the spec, design, and implementation. Design docs are maintained privately.
How did you test the PR?
Build status: PASS. Verified commands: root pnpm build (5 successful, cached), pnpm knip (exit 0), changed-file prettier --check (45 files, all matched); translations check (all complete). From src: pnpm bundle, pnpm check-types, pnpm lint (all exit 0). Emitted worker/WASM checks passed: 38 valid WASM headers, worker smoke test returned one block without fallback; asset tests: 37 passed.
Verified: npx vitest run services/code-index services/tree-sitter: 801 passed, 0 failed, 12 skipped. The 12 skips predate this change. Reported base revision: 762 passed (not rerun here).
Known limits
Not verified: packaged VSIX on remote/WSL hosts; real termination of a non-finishing WASM parse; live editor responsiveness. The 10 s / 2 s limits are provisional. Only the OpenAI embedder honours cancellation. Cancelling mid-scan before the completion phase saves no hashes. The Stopping state is no longer set on stop.
Pre-PR Code Review
Agent notes
Validation: tester and builder child tasks; review outcome and human approvals supplied by the workflow handoff. Base: b32c5cf. Scope: 45 files; private docs and protected items excluded.
Deferred Items
Reviewed commit: 8c38f1c.
Findings 1 and 2 were fixed in PR #34 and are not deferred.