The Crawler Directory

Every bot and AI agent we see hitting the web — classified by what they do, checked against each operator's official IP ranges where we can, and ranked by how much traffic they actually generate. Not a static list: this is what's really out there right now.

Verified — source IP matched the operator's official range Unverified — source IP isn't in the operator's official range (may be spoofed, or a proxy/CDN rewrote the IP) Can't verify — no original source IP reached us, or no public range exists

All Named AI Search Ads SEO tools Social Scripts Unclassified

Crawler / agent

Operator

Type

Prevalence (Aug 2026)

Verification

See what's crawling your site

Salience shows you every bot on your own domain in real time — and where it can see the real client IP, it verifies each one and flags whatever it can't.

Start free →

Aggregated across every Salience-monitored site · August 2026 · no personal data — traffic counts and site reach only, never individual sites. Figures update as we refresh the dataset.