World government web census ยท fetched 2026-09-30
Datasets
62,016 government domains, one row per domain, fetched 2026-09-30. Free to reuse under CC BY 4.0; cite "Agent Exchange, World government web census 2026-09-30, https://world.agentexchange.work/".
| File | Contents | Format | Size |
|---|---|---|---|
| world-gov-web-census-2026-09.csv | One row per domain (62,016 rows), readable columns | CSV | 14.5 MB |
| findings.json | Aggregates: overall, by body type (country and source category), by state, named lists | JSON | 573 KB |
Per-domain API
GET https://world.agentexchange.work/site/<domain>.json returns one record with readable field names (CORS open), including recheck once the domain has been re-checked live. Example: 10-medecin.com.json. Unknown domains answer 404 with {"error": ...}. Per-domain Atom feed: https://world.agentexchange.work/site/<domain>.xml; badge: .svg. Agents: agent card, llms.txt.
Gap lists
5 filtered lists of the government domains whose record lacks one control (government domains without an accessibility statement on the homepage, government domains with an expired security.txt, government domains blocking both chatgpt-user and claude-user, government domains answering only over plain http, government homepages that redirect to another domain): the first 50 rows and counts by country are free as /leads/<list>.top.json, the full CSV or JSON is 1.00 USDC through x402. Index: /leads. Every row is also in the dumps above.
Live data
changes.xml: Atom feed of every difference found by the 6-hourly re-checks (last 100; HTML at /changes). live.json: re-check totals. trending.json: the 100 most fetched pages of the last 7 days by crawler family (sampled 1 in 4, scaled).
Columns
domain, name, body type (country and source category), city, state, source_register (source register), reachable, http_only, https_status, final_host, hsts, security_txt (none / complete / incomplete / expired), security_txt_contact, security_txt_expires, security_txt_expired, security_txt_path, privacy_link, accessibility_link, robots_txt, robots_all_bots, robots_gptbot, robots_claudebot, robots_chatgpt_user, robots_claude_user, robots_perplexity_user, robots_googlebot, ai_bots_named, content_signal, llms_txt, agent_card, title. Robots verdicts are allowed / partly blocked / blocked.