crawlstate.com
crawlstate.com is a live English-language website hosted on Cloudflare, Inc. (AS13335) in the United States, served by Cloudflare. cl0q first saw it on 2026-09-06 and last crawled it on 2026-09-06. It resolves to 104.21.33.243. Its favicon is fingerprinted by SHA-256, so you can find every other site that serves the same icon.
How the site describes itself:
CrawlState — the permission layer of the AI web
A daily record of which websites allow, block, or charge AI crawlers. Query one domain or half a million, through an API built for crawler operators.
| Status | 200 OK — live & indexed by cl0q |
|---|---|
| First seen | 2026-09-06 |
| Last crawled | 2026-09-06 |
| IPv4 | 104.21.33.243 |
| IPv6 | 2606:4700:3036::ac43:c1fc |
| Network | AS13335 Cloudflare, Inc. · California |
| Language | en |
| Server | cloudflare |