perplexitybot

Eight SEO and GEO agent skills for WordPress: AI crawler log forensics, page citability scoring, brand visibility in AI answers, robots.txt and llms.txt governance, migration redirects, SEO plugin driving, and agency portfolio reporting. Every analysis ends in a self-contained HTML report.

See which AI crawlers can read your site — GPTBot, ClaudeBot, PerplexityBot & 20 more. Curated, operator-sourced bot list + a zero-dep CLI and GitHub Action to audit robots.txt, test reachability, read access logs, and gate it all in CI.

Agent-native multi-bot web crawler simulator. See your site through the eyes of Googlebot, GPTBot, ClaudeBot, and PerplexityBot.

Complete database of AI search engine crawler user-agents (GPTBot, PerplexityBot, ClaudeBot) with robots.txt configuration examples

Validates whether GPTBot, ClaudeBot, PerplexityBot and other LLM crawlers are allowed by a robots.txt file.

Visual robots.txt manager for the AI era. Block or allow GPTBot, ClaudeBot, PerplexityBot, and 20+ AI crawlers with one click.

The best AEO/GEO skill for Claude Code. Audit, fix, and monitor your website's visibility across ChatGPT, Claude, Perplexity, Gemini, and Google AI Overviews. 100+ optimization techniques, 33 evidence collectors, 4-vector GEO Score. Built on Princeton KDD 2024 paper. Free, MIT.

Four free MCP tools to check whether AI engines can find, read and recognise a website. No account, no API key, no LLM calls.

AI Observatory — verified AI and search crawler retrieval monitoring for business websites

User-agent simulator for major LLM crawlers: measures response code, content length, and JS-dependency

Record which AI crawlers read your pages. Forwards GPTBot, ClaudeBot, PerplexityBot and friends to rankveo, and nothing else.

Free no-tracking browser utility for drafting and auditing AI crawler robots.txt rules.

21 AI crawler user-agents we detect, and which 14 of them actually reached one real site over 74 days. First-party detection list plus first-party access logs. Not observed is written "not observed", never 0.

Check whether AI crawlers can access a URL (robots.txt) plus page legibility signals. Zero dependency CLI and library.

Open core of the CrawlCheck AI-crawler visibility scanner: robots.txt policy per crawler agent, user-agent group shadowing detection, and log-line forgery verification against operators' published IP feeds. Dependency-free, parity-tested against production.

Machine-readable dataset of AI crawler user-agents — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, and more. Includes robots.txt recipes and weekly verification.

Audit robots.txt for sitemap hygiene, crawl mistakes, and explicit AI crawler policy.