Top 100,000 websites census

website lookup · census · com · fetched 2026-09-30 · JSON

investopedia.com

investopedia.com · Simple Page

TLD familycom
Rank990
Website reachable over HTTPSyes (HTTP 403)
HSTS headeryes
security.txt (RFC 9116)incomplete published at /.well-known/security.txt; missing the required Expires line
Privacy policy linked from the homepagenot linked
Accessibility statement linked from the homepagenot linked
robots.txtserved; GPTBot partly blocked, ClaudeBot blocked, ChatGPT-User partly blocked, Claude-User blocked, Perplexity-User blocked, Googlebot partly blocked, all other bots partly blocked; names 13 AI crawlers explicitly
llms.txtnot published
A2A agent cardnot published

What to fix

  1. security.txt is missing the Expires line, which RFC 9116 requires. Guide
  2. robots.txt tells Claude-User not to open pages for users; if that is unintended, split training crawlers from assistants. Rules for AI crawlers
  3. No privacy policy link found on the homepage. Legal notices checklist
  4. No accessibility statement link found on the homepage. Legal notices checklist
  5. No llms.txt: a short Markdown index of what the site offers to AI assistants. Guide
  6. No A2A agent card at /.well-known/agent-card.json. Guide

Full agent-readiness grade (live)

Run this site?

Instant AI visibility report for investopedia.com: $9 · $49 full report or $99 monthly monitoring · Hosted accessibility feedback and privacy request desk

One fetch on 2026-09-30; homepage links detected from anchors and visible text, so a page that exists but is not linked from the homepage is not counted. Record as JSON: investopedia.com.json (CC BY 4.0). Not legal advice.