Explore the web

A live look at the open web as cl0q’s own crawler sees it — an independent index, no third-party search behind it. Updated 35 min ago.

18.3M
Pages indexed
23.2M
Domains known
990
Top-level domains
16M
Favicons captured
๐ŸŸข +219.9k pages crawled in the last 24h — the index updates continuously

Top hosting networks click to drill in →CSVJSON

View all 150 →

Top-level domains click to explore →CSVJSON

com61.8%
nl7.0%
org6.0%
net5.1%
de4.4%
pl2.4%
uk2.1%
cn2.1%
jp1.5%
fr1.5%
br1.4%
ru1.2%
it1.2%
info1.2%
be1.1%
View all 150 →
com50.0%
nl5.6%
org4.8%
net4.1%
de3.6%
pl1.9%
uk1.7%
cn1.7%
jp1.2%
fr1.2%
br1.1%
ru1.0%
it1.0%
info1.0%
be0.9%
ch0.8%
au0.8%
tv0.7%
ca0.7%
es0.7%
cz0.6%
eu0.6%
se0.6%
io0.5%
co0.5%
at0.5%
in0.4%
top0.4%
xyz0.4%
edu0.3%
ai0.3%
dk0.3%
shop0.3%
app0.3%
ro0.3%
hu0.3%
gr0.3%
id0.3%
fi0.3%
no0.3%
pt0.3%
za0.2%
cc0.2%
me0.2%
us0.2%
ph0.2%
online0.2%
dev0.2%
mx0.2%
sk0.2%
ir0.2%
ar0.2%
tr0.2%
nz0.2%
kr0.2%
site0.2%
cl0.1%
biz0.1%
ua0.1%
pro0.1%
gov0.1%
ie0.1%
store0.1%
vn0.1%
vip0.1%
lt0.1%
il0.1%
sbs0.1%
tw0.1%
ee0.1%
blog0.1%
my0.1%
nu0.1%
hr0.1%
si0.1%
tech0.1%
asia0.1%
club0.1%
bg0.1%
sg0.1%
live0.1%
space0.1%
rs0.1%
lv0.1%
click0.1%
ae0.1%
lol0.1%
cloud0.1%
cat0.1%
art0.1%
kz0.0%
hk0.0%
link0.0%
studio0.0%
life0.0%
by0.0%
world0.0%
is0.0%
pe0.0%
lu0.0%
work0.0%
garden0.0%
fun0.0%
one0.0%
agency0.0%
to0.0%
ma0.0%
th0.0%
design0.0%
ge0.0%
media0.0%
pk0.0%
ng0.0%
news0.0%
uz0.0%
group0.0%
ke0.0%
fm0.0%
social0.0%
ws0.0%
ba0.0%
icu0.0%
am0.0%
gg0.0%
su0.0%
cfd0.0%
games0.0%
az0.0%
md0.0%
cam0.0%
today0.0%
li0.0%
sa0.0%
cv0.0%
bet0.0%
ink0.0%
win0.0%
lk0.0%
cyou0.0%
sh0.0%
page0.0%
buzz0.0%
guide0.0%
tokyo0.0%
coop0.0%

What powers the web server software — click to drill in →CSVJSON

nginx27.2%
other22.3%
apache18.3%
iis1.5%
caddy0.3%
google0.2%
envoy0.1%

Languages page language — click to drill in →CSVJSON

en66.0%
de6.3%
nl4.7%
es4.1%
zh3.7%
fr3.7%
ja2.9%
pt2.8%
it1.8%
ru1.7%
id1.2%
pl1.2%
View all 150 →
en60.8%
de5.8%
nl4.3%
es3.8%
zh3.4%
fr3.4%
ja2.7%
pt2.6%
it1.7%
ru1.5%
id1.1%
pl1.1%
cs0.7%
tr0.6%
vi0.6%
sv0.6%
ko0.5%
ro0.4%
ar0.4%
th0.3%
da0.3%
fa0.3%
fi0.3%
el0.3%
hu0.3%
sk0.2%
zxx0.2%
uk0.2%
hi0.2%
nb0.1%
no0.1%
ca0.1%
he0.1%
bg0.1%
lt0.1%
et0.1%
hr0.1%
bn0.1%
sl0.1%
sr0.1%
cn0.0%
lv0.0%
en_us0.0%
ms0.0%
zh_cn0.0%
is0.0%
az0.0%
gl0.0%
bs0.0%
tl0.0%
de_de0.0%
jp0.0%
uz0.0%
eu0.0%
ka0.0%
ua0.0%
x0.0%
pt_br0.0%
mk0.0%
eng0.0%
se0.0%
nl_nl0.0%
mn0.0%
af0.0%
en_gb0.0%
hy0.0%
[[ routes.idioma ]]0.0%
cz0.0%
sq0.0%
nn0.0%
fr_fr0.0%
us0.0%
fil0.0%
my0.0%
lang=0.0%
ur0.0%
kk0.0%
in0.0%
'tr'0.0%
gb0.0%
ga0.0%
utf0.0%
vn0.0%
de_ch0.0%
ch0.0%
br0.0%
zh_tw0.0%
kr0.0%
default0.0%
si0.0%
pt_pt0.0%
{{ lc.language }}0.0%
am0.0%
fr_ca0.0%
cy0.0%
en_au0.0%
{{this.intl.locale}}0.0%
cs_cz0.0%
eo0.0%
dk0.0%
ru_ru0.0%
km0.0%
es_us0.0%
rs0.0%
es_es0.0%
english0.0%
"en0.0%
ba0.0%
nl_be0.0%
ta0.0%
ku0.0%
mr0.0%
sa0.0%
ne0.0%
ml0.0%
auto0.0%
sn0.0%
tt0.0%
ht0.0%
pl_pl0.0%
be0.0%
fo0.0%
ha0.0%
sk_sk0.0%
gr0.0%
pa0.0%
id_id0.0%
gor0.0%
it_it0.0%
ja_jp0.0%
ckb0.0%
sw0.0%
undefined0.0%
st0.0%
en_ca0.0%
es_mx0.0%
tw0.0%
gd0.0%
gb23120.0%
sp0.0%
en_nz0.0%
ptbr0.0%
%paraglide.lang%0.0%
lzh0.0%
frp0.0%
al0.0%
me0.0%
en_en0.0%
fa_ir0.0%
gu0.0%

Response mix live vs redirect vs deadCSVJSON

2xx85.1%
4xx12.7%
5xx1.7%
3xx0.4%
other0.0%

Biggest subdomain hubs platforms — click for the crawl profile →CSVJSON

View all 15 →

Most-shared favicons icons served by the most sites — click to pivot →

cl0q fingerprints every site’s favicon by SHA‑256. These are the icons served by the most domains — CMS defaults, hosting & parking kits, and shared‑infrastructure clusters. Click any icon to find every site serving it — the favicon: pivot behind the API.

Counts are exact, across all crawled favicons. Click any icon for the full site list, or query favicon:<sha256> via the API.

Pull the full data every matching domain, as CSV or JSON

The buttons above export what a graph shows. To pull every domain behind it — a whole TLD, network, language, server, country, or the entire index — build a full report. Large ones generate in the background; a download link appears when it’s ready. No API key, free while in beta.

Columns: domain, status, title, description, server, language, network. Reports expire after 3 days. One report generates at a time (paced) so pulls never slow the crawler.

State of the web what a typical indexed page looks like

Measured across cl0q’s own crawl — not a survey, not a top-1M list. Medians and adoption rates over a live sample of the whole index.

327ms
Median response time
469
Median words per page
8
Median outbound links
12
Median images
52.1%
Use OpenGraph tags
35.7%
Use Twitter cards
57.2%
Declare a canonical URL
66.3%
Declare UTF-8
32.8%
Reachable over IPv6
49.3%
Have mail (MX) records
51.5%
Send ETag / Last-Modified *

* measured over pages crawled in the last 24h. Everything else is a sample of the full index. Page size is deliberately omitted — the crawler isn’t recording it reliably yet.

Where the web is hosted by server location — click a country to drill in →CSVJSON

Loading world map…
View all 224 →

224 countries indexed · location is a site’s server IP, so CDN-fronted sites show at the edge.

Browse by top-level domain โ€” 574 profiled

Take the data — all of it open index, open API, no key

cl0q is an independent, open-source search engine: our own crawler, our own index, nobody else’s results behind it. Every number on this page is a JSON endpoint — no API key, no sign-up, CORS open, so you can chart it, cite it, or fork it.

Full reference in the developer docs. Counts are exact; distributions are sampled and labelled as such. If you publish something with it, we’d love to see it.

Build on it — free API