YLB ScoreAsk for yours

Part I, Can people find you?

You have told AI assistants not to read your site

Every website carries a small text file that says which automated readers are welcome. Some off-the-shelf settings turn away the readers that AI assistants use, which means when somebody asks an assistant for a business like yours, it has never seen your site and cannot mention you.

Whether to allow them is your call. Most local businesses want to be found; if that is you, this is a one-line change.

Worth about 1 point of your 100 for a typical businessAbout 5 min, by your web person

See for yourself

Open your website address with /robots.txt on the end and look for lines naming GPTBot, ClaudeBot or PerplexityBot followed by the word Disallow.

How to fix it

  1. Ask your web person to remove the lines that block those readers, or to allow them explicitly.
  2. Leave the rest of the file alone; it is doing other jobs.
The technical version, for your web person

robots.txt. We look for a User-agent group naming GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Google-Extended or CCBot with Disallow: / beneath it.

What the check reads. robots.txt parsed with a per-bot regex looking for 'Disallow: /' following that bot's User-agent line, for each name in AI_BOTS.

This is one of the checks in a YLB Score report. Every report is free, read by a person before it goes out, and written so that you can act on it without us. Ask for yours.