Skip to content

feat(registry): add six crawlers to the agent table - #141

Merged
cport1 merged 1 commit into
mainfrom
agents-sweep-20261118
Nov 18, 2026
Merged

cport1 merged 1 commit into
mainfrom
agents-sweep-20261118

Conversation

@cport1

@cport1 cport1 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Adds six crawlers from the weekly bot sweep:

  • CGMSearchBot (search, CGMIMM local business search)
  • 360Spider (search, Qihoo 360)
  • SpiderLing (commercial, Sketch Engine corpus crawler; its operator says it is not a generative-AI crawler)
  • NetAPI Bot (commercial, domain datasets)
  • AntiCloneBot (commercial, finds copies of online shops; signs requests with Web Bot Auth)
  • Hunyuan (training; links to Tencent but undocumented, so no operator is named)

Regenerated agents table and parity vectors from pkg/agents. Companion to https://github.com/WebDecoy/app/pull/1894

Regenerated agents table and parity vectors from pkg/agents.
@cport1
cport1 merged commit 51e2cbc into main Nov 18, 2026
5 checks passed
@cport1
cport1 deleted the agents-sweep-20261118 branch November 18, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant