Skip to content

Accuracy pass across every feature - #28

Merged
rushilrawat merged 3 commits into
mainfrom
fix/accuracy-pass
Nov 19, 2026
Merged

rushilrawat merged 3 commits into
mainfrom
fix/accuracy-pass

Conversation

@rushilrawat

@rushilrawat rushilrawat commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner

Requested by Rush · project thread

Summary

This PR only fixes accuracy. Nothing in the UI changes. I audited the data and output behind every feature, fixed what was wrong, and added checks so the same kinds of mistakes fail CI next time. All fixes live in scripts/expand/accuracy_pass.py, which is idempotent and comments each fix with why it was wrong.

Tech graph (seed data)

  • Implementation languages: Vite and Vue 3 are written in TypeScript, not JavaScript. Nmap is written in C++, not C.
  • Optional runtimes were marked as hard dependencies: PyTorch and TensorFlow → CUDA, and Kubernetes → containerd, are now integrates_with. CUDA is optional, and Kubernetes accepts any CRI runtime.
  • Edges that made false claims:
    • Cloudflare "supports" DDoS attacks is now related_to.
    • Falco, Trivy and Snort were linked to threat modeling. They now point at EDR, vulnerability assessment and packet sniffing.
    • Protobuf "part of the gRPC ecosystem" is now gRPC integrates_with protobuf.
    • LeRobot "part of" Transformers is now related_to.
  • Stale facts:
    • Coq was renamed Rocq in 2025. It keeps the Coq alias and gets the new URL.
    • Remix v2 merged into React Router v7.
    • OpenGL is mature (frozen at 4.6 since 2017).
    • Cuckoo Sandbox is deprecated (unmaintained since 2019).
    • ESP32 is a platform (hardware), not a developer tool. Sass is a preprocessor, not a standard.
  • Concept text: fixed the descriptions of supercomputer (thousands of nodes, not tens of thousands), gate-all-around transistors (3 nm and 2 nm), the sampling theorem (band-limited signals only) and RSC. "Shard" is renamed "Database Sharding".
  • Summaries: two Wikipedia titles pointed at other subjects. Embodied AI pointed at Embodied cognition and Fraud Detection pointed at Fraud.

Guided paths

  • Fixed six captions that overclaimed: ESP32 "on a coin-cell budget", "Bitcoin was the first blockchain", Matter "runs on top of Thread" (it also runs over Wi-Fi and Ethernet), "every brand works together", "ATT&CK names every technique", and the Let's Encrypt wording.

Famous apps

  • LinkedIn: Kafka was open-sourced in 2011. It was not created that year.
  • Figma: added WebGPU, the renderer since 2025 with WebGL as the fallback.
  • ChatGPT: Azure is still the primary cloud, plus Oracle, CoreWeave and Google Cloud since 2025.
  • Wikipedia: CirrusSearch is moving to OpenSearch (added to not_in_atlas). Please double-check this one. It comes from memory and I could not verify it from here.

Repo X-Ray scanner

  • Gradle version catalogs and plugins {} blocks are now read. Before, Android, Kotlin Multiplatform, Compose and Firebase were missed whenever they were applied through convention plugins (nowinandroid, kotlinconf-app). This adds a new gradle-plugin ecosystem in package_map.json.
  • setup.py is now parsed with ast and never executed. Before, it was selected but ignored, so huggingface/transformers missed torch and numpy.
  • Mis-mapping: huggingface-hub (the Hub client) mapped to Transformers. It is now never-map. @types/jest now maps to Jest.
  • Corpus:
    • Three new fixtures: kotlin-android-catalog, py-setup-py-lib, and the negative case neg-gradle-plugin-lookalikes.
    • huggingface-hub added to the PyPI name-collision fixture.
    • Ground truth corrected where the scanner was right and the hand-written truth was incomplete: NiA uses protobuf, and kotlinconf has an Android app, JUnit and PostgreSQL. Extras that could go either way were marked optional.
  • Evaluation: precision 1.000 → 1.000, recall 0.979 → 0.995 over 135 fixtures.

Stack recommender and role paths

  • I reviewed the catalog, synonyms and roles and found no wrong facts. New checks: the recommender never offers a deprecated or historical technology, and roles never teach one.

Validation added

  • scripts/validate_dataset.py has a new validate_semantics pass. It fails on:
    • a directional edge (built_on, written_in, depends_on, part_of_ecosystem) that runs both ways
    • alternative_to between different node types, or on the same pair as built_on, depends_on or written_in
    • a non-offensive tool that supports, implements, enables or is used_for an attack concept (this would have caught the Cloudflare edge)
  • backend/tests/test_accuracy_pass.py (17 tests): regression guards for the facts above, the semantic rules, the Gradle and setup.py parsers, and a check that accuracy_pass.py has nothing left to apply.

Layout

The edge and node counts are unchanged (1,194 / 3,517), so edge ids stay stable. I kept cross-field affinities the same, so no field moves on the ring. layout.json was regenerated: 238 nodes move a little inside their own cluster.

Not changed (worth a look later)

  • Some concept slugs don't match their names: cross-platform-ui is Declarative UI and message-broker is Publish–Subscribe. Renaming slugs would break share links and learning maps, so I left them.
  • frontend/public/example-scans/ were not regenerated. That needs repo access through the GitHub API or git, and the Gradle and setup.py improvements will show up the next time they are rebuilt.

Test plan

  • pytest (backend): 272 passed
  • vitest: 160 passed, tsc --noEmit clean
  • scripts/validate_dataset.py: OK
  • Scanner evaluation harness: precision 1.000, recall 0.995

https://claude.ai/code/session_01EMFEEMXAuR2wLnpNEoZQFz


Generated by Claude Code

Data: correct implementation languages (Vite/Vue -> TypeScript, Nmap -> C++,
TensorFlow -> C++), drop hard "depends_on" edges for optional runtimes (CUDA,
containerd), fix edges that read as wrong claims (Cloudflare "supports" DDoS
attacks, Falco/Trivy/Snort as threat modeling, k6 as integration testing,
protobuf as part of gRPC, Sigma.js on WebGPU), update stale facts (Coq ->
Rocq, Remix -> React Router v7, OpenGL frozen at 4.6, Cuckoo deprecated,
ESP32 is hardware) and two Wikipedia titles that pointed at other subjects.
Guided path captions and famous-app notes corrected (Kafka open-sourced 2011,
Figma on WebGPU, OpenAI's extra clouds, Wikipedia's OpenSearch move).

Validator: new semantic rules (directional edges both ways, alternative_to
across node types or alongside built_on/depends_on, non-offensive tools that
"support" an attack).

Scanner: Gradle version catalogs and plugins blocks are read (Android,
Kotlin Multiplatform, Compose, Firebase now detected), setup.py is parsed with
ast, huggingface-hub no longer maps to Transformers, @types/jest maps to Jest.
Three new corpus fixtures; ground truth corrected where the scanner was right.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMFEEMXAuR2wLnpNEoZQFz
Revert the TensorFlow, Sigma.js and k6 edge changes (each was defensible as
written) and retype the PyTorch-CUDA edge rather than retarget it, so field
affinities stay the same and no whole field moves on the ring. Falco now links
to EDR inside the security field. layout.json regenerated: 238 nodes shift
within their own clusters, no field changes place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMFEEMXAuR2wLnpNEoZQFz
@rushilrawat rushilrawat self-assigned this Nov 19, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMFEEMXAuR2wLnpNEoZQFz
@rushilrawat
rushilrawat merged commit f583f10 into main Nov 19, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants