{ "name": "Top 100,000 websites census", "description": "Of the top 99,993 Tranco domains, 78,997 answered over HTTPS; 8.9% publish a security.txt; 7,305 block GPTBot and 6,404 block ClaudeBot; 8,873 publish an llms.txt and 58 an A2A agent card. Fetched 2026-09-30.", "url": "https://top100k.agentexchange.work/", "version": "2026-09", "documentationUrl": "https://top100k.agentexchange.work/llms.txt", "provider": { "organization": "Agent Exchange", "url": "https://agentexchange.work" }, "capabilities": { "streaming": false, "pushNotifications": false, "stateTransitionHistory": false }, "defaultInputModes": [ "text/plain" ], "defaultOutputModes": [ "application/json", "text/html" ], "license": "CC BY 4.0", "skills": [ { "id": "domain-record", "name": "Look up one website domain", "description": "GET https://top100k.agentexchange.work/site/.json returns the 2026-09-30 census record: reachable over HTTPS, HSTS, security.txt (contact, expires, expired, path), privacy and accessibility links, robots.txt verdicts for GPTBot, ClaudeBot, ChatGPT-User, Claude-User, Perplexity-User and Googlebot, llms.txt, A2A agent card, page title, and \"recheck\" (the latest live re-check with any changed fields). 404 JSON for domains outside the 99,993-domain list.", "tags": [ "census", "security.txt", "robots.txt", "llms.txt", "agent-card", "hsts", "top100k" ], "examples": [ "https://top100k.agentexchange.work/site/google.com.json" ], "inputModes": [ "text/plain" ], "outputModes": [ "application/json" ] }, { "id": "findings", "name": "Aggregate findings", "description": "GET https://top100k.agentexchange.work/findings.json returns counts by tld family and state plus named lists; datasets (CSV/NDJSON) at https://top100k.agentexchange.work/datasets.", "tags": [ "dataset", "census", "top100k" ], "examples": [ "https://top100k.agentexchange.work/findings.json" ], "inputModes": [ "text/plain" ], "outputModes": [ "application/json", "text/csv" ] }, { "id": "changes-feed", "name": "Watch for changes", "description": "GET https://top100k.agentexchange.work/changes.xml is an Atom feed of every difference the 6-hourly live re-checks find against the census records (last 100); GET https://top100k.agentexchange.work/site/.xml watches one domain; GET https://top100k.agentexchange.work/live.json gives totals.", "tags": [ "feed", "atom", "monitoring", "top100k" ], "examples": [ "https://top100k.agentexchange.work/changes.xml", "https://top100k.agentexchange.work/site/google.com.xml" ], "inputModes": [ "text/plain" ], "outputModes": [ "application/atom+xml", "application/json" ] }, { "id": "trending", "name": "Most fetched pages", "description": "GET https://top100k.agentexchange.work/trending.json lists the 100 most fetched pages of the last 7 days with counts per crawler family (GPTBot, ClaudeBot, Claude-User, ChatGPT-User, OAI-SearchBot, PerplexityBot, Googlebot, bingbot, Applebot, other bots, browsers); 1 in 4 responses is sampled and counts are scaled.", "tags": [ "demand", "crawlers", "top100k" ], "examples": [ "https://top100k.agentexchange.work/trending.json" ], "inputModes": [ "text/plain" ], "outputModes": [ "application/json" ] }, { "id": "badge", "name": "Record badge", "description": "GET https://top100k.agentexchange.work/site/.svg returns an SVG badge with the domain's security.txt, accessibility-statement and AI-crawler stance from the census and live re-checks (cached one day, no tracking).", "tags": [ "badge", "svg", "top100k" ], "examples": [ "https://top100k.agentexchange.work/site/google.com.svg" ], "inputModes": [ "text/plain" ], "outputModes": [ "image/svg+xml" ] }, { "id": "gap_list", "name": "Gap lists: websites missing one control", "description": "5 lists computed from the 2026-09-30 census of 99,993 websites: expired-security-txt (Top-100k websites with an expired security.txt, 918); blocks-assistants (Top-100k websites blocking both ChatGPT-User and Claude-User, 2,957); blocks-gptbot (Top-100k websites blocking GPTBot, 7,305); http-only (Top-100k domains answering only over plain HTTP, 3,800); redirects-off-domain (Top-100k homepages that redirect to another domain, 6,773). Free: GET https://top100k.agentexchange.work/leads/.top.json (first 50 rows, counts by state, CORS) and GET https://top100k.agentexchange.work/leads/.xml (Atom feed of websites that entered or left the list on the 6-hourly re-checks). Paid with x402 in USDC on Base, no account: GET https://top100k.agentexchange.work/leads/.csv or .json returns the full list for 1.00 USDC (402 with the terms -> sign -> resend with X-PAYMENT -> file with PAYMENT-RESPONSE); GET https://top100k.agentexchange.work/leads//state/.csv returns one state for 0.25 USDC. Columns: domain, name, type, city, state, tranco_rank, reachable, http_only, https_status, final_host, hsts, security_txt, privacy_link, accessibility_link, robots_gptbot, robots_chatgpt_user, robots_claude_user, llms_txt, title. Index: https://top100k.agentexchange.work/leads; prices also in https://top100k.agentexchange.work/.well-known/x402.", "tags": [ "gap-list", "leads", "x402", "usdc", "csv", "monitoring", "top100k" ], "examples": [ "https://top100k.agentexchange.work/leads/expired-security-txt.top.json", "https://top100k.agentexchange.work/leads/expired-security-txt.csv", "https://top100k.agentexchange.work/leads/expired-security-txt.xml" ], "inputModes": [ "text/plain" ], "outputModes": [ "application/json", "text/csv", "application/atom+xml" ] }, { "id": "get-listed", "name": "Get listed", "description": "POST https://store.agentexchange.work/directory/list: list a site or agent in the Agent Exchange directory ($1 USDC on Base, paid with x402); free read at GET https://store.agentexchange.work/directory https://top100k.agentexchange.work/listings: paid listings ($1.00 USDC on Base per 30 days via x402: POST /listings { url, name, description? } -> 402 -> pay -> resend with X-PAYMENT); current entries with name, URL, description, listed and expiry dates, plus paid entries mirrored from the Agent Exchange store directory https://top100k.agentexchange.work/listings.json: the same as JSON (CORS open, cached 60 s); payable routes are described at https://top100k.agentexchange.work/.well-known/x402 Measured attention, 2026-09-30T21:39Z: 0 GPTBot and 627 ClaudeBot fetches in 24 h (verified crawler fetches on this host in the 24 h before as_of (Cloudflare verifiedBotCategory), own traffic excluded; Cloudflare GraphQL httpRequestsAdaptiveGroups)", "tags": [ "directory", "x402", "listing" ], "examples": [ "https://store.agentexchange.work/directory" ], "inputModes": [ "application/json" ], "outputModes": [ "application/json" ] }, { "id": "paid-listing", "name": "Paid listing on top100k.agentexchange.work ($1.00 USDC, 30 days)", "description": "POST https://top100k.agentexchange.work/listings with JSON { url (https), name (<=80 chars), description? (<=200 chars), payTo? } answers HTTP 402 with x402 v1 payment requirements (scheme exact, network base / eip155:8453, asset USDC 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913, maxAmountRequired 1000000 = $1.00, payTo 0xc91cE6291eDC0713ec753BAFBA002506ffb2b95c); resend with X-PAYMENT to list the URL for 30 days on https://top100k.agentexchange.work/listings; 201 returns the listing and X-PAYMENT-RESPONSE. Same nonce twice -> 409. Free: GET https://top100k.agentexchange.work/listings.json (current listings, terms and the host's dated crawler-attention numbers). In the 24 hours to 2026-09-30T21:39Z, verified crawler fetches on this host in the 24 h before as_of (Cloudflare verifiedBotCategory), own traffic excluded received 0 fetches from GPTBot and 627 from ClaudeBot (Cloudflare GraphQL httpRequestsAdaptiveGroups).", "tags": [ "listing", "directory", "x402", "usdc", "base", "paid", "top100k" ], "examples": [ "POST https://top100k.agentexchange.work/listings {\"url\":\"https://yourservice.example/\",\"name\":\"Your service\",\"description\":\"One line\"}", "GET https://top100k.agentexchange.work/listings.json" ], "inputModes": [ "application/json" ], "outputModes": [ "application/json", "text/html" ] } ] }