Top 100,000 websites census

Top 100,000 websites census · gap lists · census 2026-09-30 · JSON (top 50) · feed

Top-100k homepages that redirect to another domain

6,773 of the 99,993 websites in the census list (6.8%; 8.6% of the 78,997 reachable over HTTPS), as fetched on 2026-09-30.

On this list

6,773

of 99,993 websites

Share of the list

6.8%

8.6% of reachable sites

Full list

1.00 USDC

x402, per file; first 50 rows free

What this gap means

The homepage redirected to a host outside the listed domain (a different registrable name) on the fetch date.

Control: Homepage served on the listed domain. Rule: redirects_off_domain over the census record (reachable sites only). Who buys this: Web agencies, domain and brand-protection vendors, security teams checking for stale redirects.

This list maps to no legal obligation in this census; it is a gap in published practice.

Free: the first 50 of 6,773 (by rank)

DomainTLD familyRankCensus record
amazonaws.comcom6HTTP 200 via aws.amazon.com · HSTS yes · security.txt expired · privacy no · accessibility no · GPTBot partly blocked
fbcdn.netnet13HTTP 200 via www.facebook.com · HSTS yes · security.txt complete · privacy no · accessibility no · GPTBot partly blocked
twitter.comcom15HTTP 200 via x.com · HSTS yes · security.txt expired · privacy yes · accessibility yes · GPTBot blocked
dzen.ruRU17HTTP 200 via sso.passport.yandex.ru · HSTS yes · security.txt none · privacy no · accessibility no · GPTBot blocked
office.comcom22HTTP 200 via m365.cloud.microsoft · HSTS yes · security.txt none · privacy yes · accessibility yes
hicloudcam.comcom24HTTP 200 via www.ezviz.com · HSTS no · security.txt incomplete · privacy yes · accessibility no · GPTBot partly blocked
azure.comcom28HTTP 200 via azure.microsoft.com · HSTS yes · security.txt none · privacy no · accessibility no · GPTBot partly blocked
whatsapp.netnet32HTTP 200 via www.whatsapp.com · HSTS yes · security.txt complete · privacy yes · accessibility no · GPTBot blocked
fastly.netnet35HTTP 200 via www.fastly.com · HSTS yes · security.txt none · privacy yes · accessibility no · GPTBot partly blocked
doubleclick.netnet38HTTP 200 via marketingplatform.google.com · HSTS no · security.txt none · privacy yes · accessibility no · GPTBot blocked
youtu.beBE46HTTP 200 via www.youtube.com · HSTS yes · security.txt incomplete · privacy no · accessibility no · GPTBot partly blocked
skype.comcom51HTTP 200 via teams.live.com · HSTS yes · security.txt none · privacy yes · accessibility no · GPTBot partly blocked
googledomains.comcom58HTTP 200 via domains.google · HSTS no · security.txt none · privacy yes · accessibility no · GPTBot allowed
googlesyndication.comcom60HTTP 200 via www.google.com · HSTS no · security.txt complete · privacy yes · accessibility no · GPTBot partly blocked
wa.meME69HTTP 200 via api.whatsapp.com · HSTS yes · security.txt none · privacy yes · accessibility no · GPTBot blocked
zoom.usUS74HTTP 200 via www.zoom.com · HSTS yes · security.txt none · privacy yes · accessibility no · GPTBot partly blocked
workers.devdev80HTTP 200 via www.cloudflare.com · HSTS yes · security.txt incomplete · privacy no · accessibility no · GPTBot allowed
cloudflare-dns.comcom83HTTP 200 via one.one.one.one · HSTS no · security.txt none · privacy yes · accessibility no · GPTBot allowed
nginx.comcom96HTTP 200 via www.f5.com · HSTS yes · security.txt complete · privacy yes · accessibility no · GPTBot allowed
windows.comcom97HTTP 200 via www.microsoft.com · HSTS yes · security.txt expired · privacy yes · accessibility yes · GPTBot partly blocked
t.meME103HTTP 200 via telegram.org · HSTS yes · security.txt none · privacy yes · accessibility no
blogspot.comcom104HTTP 200 via www.blogger.com · HSTS no · security.txt none · privacy yes · accessibility no · GPTBot partly blocked
actify.nlNL107HTTP 200 via www.zilverenkruis.nl · HSTS yes · security.txt complete · privacy no · accessibility no · GPTBot partly blocked
b-cdn.netnet108HTTP 200 via bunny.net · HSTS no · security.txt none · privacy no · accessibility no · GPTBot allowed
bit.lyLY117HTTP 200 via bitly.com · HSTS yes · security.txt incomplete · privacy yes · accessibility yes · GPTBot allowed
amazonvideo.comcom118HTTP 200 via www.primevideo.com · HSTS yes · security.txt none · privacy no · accessibility no · GPTBot partly blocked
google-analytics.comcom119HTTP 200 via marketingplatform.google.com · HSTS no · security.txt none · privacy yes · accessibility no · GPTBot blocked
outlook.comcom121HTTP 200 via outlook.live.com · HSTS yes · security.txt none · privacy no · accessibility no
github.ioIO127HTTP 200 via pages.github.com · HSTS no · security.txt none · privacy yes · accessibility no · GPTBot allowed
unity3d.comcom140HTTP 200 via unity.com · HSTS no · security.txt none · privacy no · accessibility no · GPTBot partly blocked
discord.ggGG142HTTP 200 via discord.com · HSTS yes · security.txt complete · privacy yes · accessibility no · GPTBot partly blocked
xiaomi.comcom158HTTP 403 via www.mi.com · HSTS no · security.txt none · privacy no · accessibility no
lencr.orgorg163HTTP 200 via letsencrypt.org · HSTS yes · security.txt complete · privacy yes · accessibility no · GPTBot allowed
adobe.ioIO165HTTP 200 via developer.adobe.com · HSTS yes · security.txt none · privacy no · accessibility no · GPTBot partly blocked
adtrafficquality.googlegoogle172HTTP 200 via www.google.com · HSTS no · security.txt complete · privacy yes · accessibility no · GPTBot partly blocked
one.oneone173HTTP 200 via www.one.com · HSTS yes · security.txt incomplete · privacy no · accessibility no · GPTBot partly blocked
vkuserphoto.ruRU178HTTP 418 via ps.userapi.com · HSTS yes · security.txt none · privacy no · accessibility no
hosting24.comcom183HTTP 200 via www.hostinger.com · HSTS no · security.txt complete · privacy no · accessibility no · GPTBot partly blocked
macromedia.comcom184HTTP 200 via www.adobe.com · HSTS yes · security.txt complete · privacy no · accessibility no · GPTBot partly blocked
paytmfirstgames.comcom186HTTP 200 via www.firstgames.in · HSTS yes · security.txt none · privacy no · accessibility no
telekom.netnet199HTTP 200 via www.telekom.com · HSTS yes · security.txt none · privacy yes · accessibility no · GPTBot partly blocked
rubiconproject.comcom220HTTP 200 via www.magnite.com · HSTS no · security.txt none · privacy yes · accessibility yes · GPTBot allowed
vungle.comcom225HTTP 200 via liftoff.ai · HSTS no · security.txt none · privacy yes · accessibility no · GPTBot partly blocked
pages.devdev234HTTP 200 via www.cloudflare.com · HSTS yes · security.txt incomplete · privacy no · accessibility no · GPTBot allowed
comcast.netnet238HTTP 403 via www.xfinity.com · HSTS yes · security.txt none · privacy no · accessibility no · GPTBot partly blocked
meraki.comcom251HTTP 200 via www.cisco.com · HSTS yes · security.txt complete · privacy yes · accessibility yes · GPTBot allowed
gmail.comcom253HTTP 200 via accounts.google.com · HSTS yes · security.txt none · privacy no · accessibility no · GPTBot partly blocked
wixsite.comcom275HTTP 200 via www.wix.com · HSTS yes · security.txt incomplete · privacy no · accessibility no · GPTBot partly blocked
crashlytics.comcom279HTTP 200 via firebase.google.com · HSTS yes · security.txt none · privacy no · accessibility no · GPTBot partly blocked
mzstatic.comcom297HTTP 200 via www.apple.com · HSTS yes · security.txt complete · privacy yes · accessibility yes · GPTBot partly blocked

The same rows as JSON: redirects-off-domain.top.json (CORS open, cached one day).

Get the full list: 6,773 rows, 1.00 USDC

x402 on Base, paid per file, no account or key. GET https://top100k.agentexchange.work/leads/redirects-off-domain.csv or GET https://top100k.agentexchange.work/leads/redirects-off-domain.json answers 402 Payment Required with the terms (scheme exact, USDC on Base, maxAmountRequired 1000000, payTo 0xc91cE6291eDC0713ec753BAFBA002506ffb2b95c); an x402 client signs the EIP-3009 authorization and resends the same request with X-PAYMENT; the file comes back with a PAYMENT-RESPONSE header. HEAD gets the same 402.

curl -sS -D - https://top100k.agentexchange.work/leads/redirects-off-domain.csv   # 402 with the x402 terms; pay and resend with X-PAYMENT

Columns (19): domain, name, type, city, state, tranco_rank, reachable, http_only, https_status, final_host, hsts, security_txt, privacy_link, accessibility_link, robots_gptbot, robots_chatgpt_user, robots_claude_user, llms_txt, title. CSV: header row, every cell quoted, yes/no for booleans. JSON: { list, title, count, by_state, columns, rows[][] }. Sizes: CSV 1.2 MB, JSON 1.2 MB. The same facts are in the free CC BY 4.0 dumps: the price buys the filtered, state-sliceable file built from the 2026-09-30 census, not exclusive data. Settlement is final; no refunds. Prices in /.well-known/x402 equal the challenges. Terms · pricing.

Card checkout

Pay with a card through Stripe and get the file on the next page: Full list CSV, $29 · JSON, $29 · all 5 lists, $79. Agents pay less with x402 (see above).

Monitor this list (free)

Atom feed: websites that entered or left this list according to the live re-checks (every 6 hours a rotating slice, each domain at most once per 14 days; each entry says which way it moved and which field changed). The re-checks do not compare redirect targets, so this feed only carries websites that stopped or started answering over HTTPS. redirects-off-domain.top.json: the first 50 rows and the counts by state as JSON, CORS open. Fields watched: reachable.

Membership reflects the homepage and well-known files as fetched on 2026-09-30: a control that exists but was not found where the census looked counts as missing, and sites answering 403 to our one polite request count as reachable. Every domain links to its record with what to fix, a JSON twin, a badge and a changes feed. Data licence CC BY 4.0. Not legal advice.