Free micro-tool
Can AI crawlers see your site?
Enter your URL and find out instantly whether ChatGPT, Gemini, Perplexity, and Claude's crawlers are allowed to index your content — and whether you have an llms.txt file.
Why AI crawler access matters
AI search is replacing Google clicks
ChatGPT, Perplexity, and Gemini now answer millions of queries directly. If their crawlers can't read your site, you're not in the answer.
robots.txt can silently block you
A single wildcard Disallow rule — often added by a security plugin or SEO tool — can block every AI crawler without you knowing.
llms.txt is the new sitemap for AI
llms.txt is an emerging standard that tells AI models what your site is about, which pages to prioritise, and how to represent your brand.
Crawler access is table stakes for GEO
Generative Engine Optimisation starts at step zero: making sure AI can crawl and read your content before scoring anything else.
Crawler access is just the start
Even if crawlers can reach your site, AI models may still ignore it. The full GEO audit scores your content structure, factual density, freshness, and more — then gives you a prioritised list of exactly what to fix.
Run the free full audit