robots.txt blocks AI answer engines
Your robots.txt blocks every major AI answer engine we check (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot and 2 more). They are instructed not to read your site at all.
- OAI-SearchBot (OpenAI, ChatGPT search results): Disallow: / (line 2, group *)
- ChatGPT-User (OpenAI, Pages ChatGPT opens for a user): Disallow: / (line 2, group *)
- Claude-SearchBot (Anthropic, Claude search results): Disallow: / (line 2, group *)
- Claude-User (Anthropic, Pages Claude opens for a user): Disallow: / (line 2, group *)
Why it matters: robots.txt is the first file AI crawlers read. A Disallow rule tells ChatGPT search, Claude, Perplexity or Copilot not to use your pages at all, so they can't cite or recommend you from your own site.