usefulweb.site AI Website Readiness Explorer

AI infrastructure & agent explorers

AI Website Readiness Explorer

Can ChatGPT, Claude, Perplexity and AI agents read your website? Explainable score.

Try an example: Blog that blocks AI training but allows search (offline) · JavaScript-only app shell (offline)

How it works

Checks a page the way AI crawlers and agents see it: robots.txt rules for 16 AI and search bots (training vs. search vs. user-triggered, RFC 9309 matching, Content-Signal), meta robots, whether the content exists without JavaScript, semantic structure, JSON-LD types, llms.txt, sitemap, OpenAPI/API catalog and machine-readable alternates. Every check is listed with its points, so the score is fully explainable.

  • robots.txt is evaluated per RFC 9309 for the page path: the group naming the bot exactly is used, otherwise the * group; the longest matching rule wins (Allow on ties). Content-Signal lines are reported.
  • Access score: search and user-request bots allowed earns the access points; blocking training-only bots (GPTBot, CCBot, Google-Extended, …) is a legitimate choice and is NOT penalised.
  • Content: visible text without JavaScript (scripts/styles removed); ≥ 500 characters = server-rendered.
  • Score = earned points / possible points × 100 over all checks (each check lists its points). Grade: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, else F.
  • Online mode fetches the page, /robots.txt, /llms.txt, /sitemap.xml, /openapi.json and /.well-known/api-catalog (RFC 9727) once each.

For scripts and AI agents

  • JSON API: POST https://aiwebsitereadinessexplorer.usefulweb.site/api/v1/calculate (same calculation as this page)
  • Interface description: OpenAPI 3.1 · AI guide: llms.txt
  • MCP server (Streamable HTTP): https://aiwebsitereadinessexplorer.usefulweb.site/mcp · tool name ai_website_readiness_explorer
Example request
curl -s -X POST https://aiwebsitereadinessexplorer.usefulweb.site/api/v1/calculate \
  -H 'content-type: application/json' \
  -d '{…see OpenAPI examples…}'

Related tools

  • DNS + Certificate Explorer – DNS records, e-mail security, CAA, live TLS certificate and CT history for a domain.
  • MCP Server Explorer – Inspect any MCP server before you connect an AI agent: tools, permissions, risks, changes over time.
  • Accessibility Snapshot – Static WCAG 2.2 snapshot of a page: findings by severity, evidence and manual checks.
  • Explorer of Everything – Search transactions, contracts, domains, packages, advisories, MCP servers and websites in one box.
Feedback