Our crawler
WhiteTieCheck crawler
November 2026
When someone runs a check on a website, we make a small number of requests to that site to read publicly available information. Requests identify themselves as:
WhiteTieCheck/1.0 (+https://whitetie.ai/bot)
What it fetches
- Only the website submitted for the check: the homepage, an About page,
robots.txtandllms.txt. - It never fetches competitors' sites, never runs JavaScript and never tries to get around a block, challenge or rate limit.
- It respects
robots.txtrules forWhiteTieCheck.
Blocking it
Add this to your robots.txt:
User-agent: WhiteTieCheck
Disallow: /If your site blocks our request, the check reports that as a finding, because AI assistants may be blocked too.
Questions: hello@whitetieseo.com.