Skip to content

fix(seo): preserve indexing rules in Markdown responses - #669

Merged
harlan-zw merged 1 commit into
mainfrom
fix/crawl-audit-boundaries
Oct 21, 2026
Merged

harlan-zw merged 1 commit into
mainfrom
fix/crawl-audit-boundaries

Conversation

@harlan-zw

Copy link
Copy Markdown
Collaborator

🔗 Linked issue

SEO check-in following #661 and #662.

❓ Type of change

  • 📖 Documentation
  • 🐞 Bug fix
  • 👌 Enhancement
  • ✨ New feature
  • 🧹 Chore
  • ⚠️ Breaking change

📚 Description

Markdown responses allowed indexing for excluded Skills and Repository hubs whose HTML pages returned noindex.
Direct Skill sources now use the existing admission rule; HTML conversion carries the rendered robots directive into Markdown.
The recovery brief records the dated follow-up and the previous repair's production result.

Existing indexing rules carried into direct and converted Markdown responses

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

@harlan-zw harlan-zw added the harlan-agent-review Pull request triage requires an adversarial Review. Adding this label manually always forces one. label Oct 21, 2026
@harlan-github-agent harlan-github-agent Bot added harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. and removed harlan-agent-review Pull request triage requires an adversarial Review. Adding this label manually always forces one. labels Oct 21, 2026
@harlan-zw

Copy link
Copy Markdown
Collaborator Author

🤖 Harlan Agent Kit Agent assessment of head 2adeebe.

The Service queued this Review behind 14 Review or Repair Tasks with four free host slots.
Its exact-head stop returned Stopped before independent review began.

The independent review found no defects. Confidence: 95/100.
The deduction covers limited built Worker fixtures. Probe, aggregator, and empty-admission variants use H3 unit coverage.
The reviewer independently reran 18 tests and traced both hooks through Nuxt AI Ready 3.3.1.
The review covered bounded SQL, admission rules, missing-source fallback, Repository ambiguity, the diagram, and evidence clocks.

Manual proof outside CI: nine built Worker HTTP checks passed with public fixture data in local D1.
Excluded Skill and Repository Markdown retained noindex. Admitted Skill and homepage responses retained index.
Markdown canonical response headers remained intact, and the reference-file view still returned HTML.
No production D1 writes ran. Production verification of this repair requires deployment.

CI passed for this exact head, including build, tests, typecheck, lint, and the harness and declared-Skills checks.
The full local suite passed 4,051 tests. Eight regression assertions failed before the repair.

This independent assessment leaves the Service Review stopped. Harlan retains the merge decision.

@harlan-zw
harlan-zw merged commit 5849608 into main Oct 21, 2026
7 checks passed
@harlan-zw
harlan-zw deleted the fix/crawl-audit-boundaries branch October 21, 2026 06:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant