Last spotted in 2022
Bot name First spotted Highest recorded
Adsbot 2022-08-18 N1KF 2022-08-18 15 N1KF
AhrefsBot 2014-08 Coolio 2022-11-04 30 N1KF
Apache HttpClient 2022-10-12 N1KF 2022-10-12 4 N1KF
BaiduSpider 2014-08-08 Coolio 2014-08-27 13 Ratburntro44
Barkrowler 2022-10-15 N1KF 2022-10-15 7 N1KF
Bingbot 2014-08 Coolio 2014-08-22 35 Colon
BLEXBot 2022-05-21 N1KF 2020-05-29 9 N1KF
Dataprovider.com 2022-10-15 N1KF 2022-10-15 9 N1KF
DotBot 2014-08-07 N1KF 2020-05-13 15 N1KF
DuckDuckGo Favicons 2015-02-01 N1KF 2020-04-19 15 N1KF
Expanse a 2022-10-15 N1KF 2022-10-15 2 N1KF
FeedFetcher Google 2022-05-21 N1KF
Googlebot 2014-08-07 Bimps1002 2020-04-19 26 N1KF
Lynx 2022-05-07 N1KF 2022-05-07 3 N1KF
MJ12bot 2014-08 Coolio 2022-12-25 38 N1KF
Opera 2022-09-16 N1KF
PetalBot 2021-01-27 archive.org 2022-12-25 34 N1KF
SemrushBot 2015-02-18 Fradeshan 2022-11-16 26 N1KF
serpstatbot 2022-05-28 N1KF 2022-12-25 39 N1KF
Sogou web 2022-03-31 N1KF 2022-06-02 8 N1KF
Unknown 2014-11-04 N1KF 2022-07-10 4 N1KF
YandexBot 2014-08-07 Different55 2022-07-06 17 N1KF
Last spotted in 2015-2021
Bot name First spotted Highest recorded Last spotted
Apple pubsub 2015-02-07 Duck 2015-02-12
archive.org 2016-02-22 archive.org 2016-02-22 5 archive.org 2021-12-30
AspiegelBot 2020-04-19 N1KF 2020-04-19 78 N1KF
CCBot 2020-08-12 archive.org 2020-10-28 13 archive.org 2021-04-11
goobot 2014-08-27 Colon 2014-08-27 2 Colon 2017-10-06
GrapeshotCrawler 2014-09-30 N1KF 2015-01-29 10 Different55
ia archiver 2014-10-14 N1KF 2019-02-15 25 archive.org 2019-08-18
linkdexbot 2015-11-07 N1KF 2018-01-31 10 N1KF
MauiBot 2018-06-23 N1KF 2021-04-11 16 archive.org
Qwantify 2021-12-30 archive.org 2021-12-30 14 archive.org
TrueBot 2015-01-19 N1KF
Windows 2015-02-16 Fradeshan
XoviBot 2015-03-02 Fradeshan
Last spotted in 2014
Bot name First spotted Highest recorded Last spotted
008 2014-10-05 Colon
A6 Indexer 2014-09-29 Coolio 2014-11-20 7 N1KF
abond Nutch 2014-10-23 N1KF 2014-10-23 2 N1KF
ADmantX Platform 2014-09-29 Coolio 2014-12-26 9 N1KF
AnyApexBot 2014-10-11 Colon 2014-10-11 2 Colon
AppEngine Google 2014-08-19 Coolio 2 Coolio
BotLogger 2014-10-28 Colon
btbot 2014-10-11 Colon
crawlbot 2014-10-14 Colon 2014-10-14 3 Colon
Curious George 2014-09-27 N1KF 2014-10-18
curl 2014-09-22 Coolio
ELinks 2014-11-20 N1KF
en Windows 2014-11-20 N1KF 2014-11-20 4 N1KF
Exabot 2014-08-24 Colon 2014-09-23 7 N1KF
Googlebot-Image 2014-10-11 Colon 2014-10-11 7 Colon
HappyFunBot 2014-10-11 Colon
hivaBot hivaBot 2014-08-11 Silverheart
iaskspider 2014-10-11 Colon
Icarus6 2014-10-19 N1KF
Konqueror 2014-08-25 Coolio 3 Coolio
LSSRocketCrawler 2014-08-19 Fradeshan 2014-08-19 2 Fradeshan
Mail.RU 2014-08-12 Silverheart 2014-09-27 6 N1KF
Mediapartners Google 2014-08-26 Ratburntro44 2014-09-22 12 N1KF
MSIE8.0 Windows 2014-08-08 Fradeshan
NetcraftSurveyAgent 2014-09-12 Coolio 2014-09-12 8 Coolio
NetSeer 2014-09-29 Coolio 2014-09-30 8 Coolio
Nintendo 3DS 2014-09-15 Coolio
nutch 1.4 2014-11-08 Coolio
Opera 2014-11-20 N1KF 2014-11-20 4 N1KF
proximic 2014-09-29 Coolio 2014-12-25 6 Different55
php system 2014-10-17 N1KF 2014-10-17 3 N1KF 2014-10-21
rogerbot 2014-09-30 N1KF 2014-09-30 2 N1KF
Scrubby 2014-10-11 Colon
SEOkicks Robot 2014-12-09 N1KF
SeznamBot 2014-08-09 N1KF 2014-09-24 10 N1KF
SiteCheck 2014-08-13 Silverheart
SMTBot 2014-10-15 N1KF 2014-10-15 2 N1KF
Turnitinbot 2014-10-11 Colon
voltron 2014-09-29 Coolio
WebCrawler 2014-10-28 some man 2014-10-28 4 some man
WeSEE Ads 2014-09-29 Coolio
Wget 2014-09-30 N1KF 2014-09-30 3 N1KF
Windows U 2014-08-08 Silverheart 2014-09-24 8 N1KF
Xenu Link 2014-10-04 N1KF 2014-10-04 8 N1KF
YisouSpider 2014-08-16 skullz17 2014-08-16 6 Fradeshan
Since the original post hasn't been updated since like 2015, I'm making my own list
Also, I added in Different55's record of 6 for proximic, since it seems Coolio missed that one.
edit: Spotted 14 SemrushBots.
edit: Added a "last spotted" column, and a section for bots spotted in 2022. I replaced Ratburntro's record of 12 Baiduspiders with 13.
edit: Fixed the 10 Grapeshots record being listed as mine and not Different's, and some formatting inconsistency.
edit: Spotted 15 PetalBots.
edit: Spotted 6 Sogou web.
edit: Spotted MJ12bot.
edit: Spotted BaiduSpider.
Last edited by N1KF (2022-12-24 20:33:57)