Thread Like Summary
Gizmo, isaac, Morgan
Total Likes: 8
Original Post (Thread Starter)
by isaac
isaac
Robots (Crawlers) List for UBB.threads - UPDATED 2020-09-24
formerly named "Search Engine Spiders List for UBB.threads"

About:
The advantage to using this list is that Search Engine Spiders get put into the correct "Search Spiders" group when viewing the forums/ubbthreads.php/online page.

This translation list is not used anywhere else in UBB.threads except on the "ubbthreads.php/online page". Having a long list of Search Engine Spiders will not slow down your forums -- the only exception is that depending on the server in which your site is hosted on, there may be a minimal delay in generating the "ubbthreads.php/online " page.

Details:
Total Agent Strings: 681
Sources:
https://udger.com/resources/ua-list/crawlers (2020-09-24) and
https://github.com/monperrus/crawler-user-agents (2020-05-19)

I've simply converted, combined, and cleaned the data for use with UBB.threads.

How-To:
UBB.threads 7.7.4 and newer -
1) COPY/PASTE the newest list to the "Robots (Crawlers)" box at: Control Panel > Display Options > Who's Online Settings
2) Click "Update General Display Options" button.
done.

Older versions of UBB.threads -
1) COPY/PASTE the newest list to the "Search Engine Agents" box at: Control Panel > Display Options > Who's Online
2) Click "Update General Display Options" button.
done.

Having problems? REMOVE any blank lines from the top/bottom of your copied list.

Fresh Installs of UBB.threads 7.7.5+ are pre-populated with this list.
The 2020-09-24 List:
Code
13TABS=13TABS
1stwarning bot=1stwarning bot
2GDPR bot=2GDPR bot
2gisbot=2gisbot
360Spider=360Spider
360spider=360spider
3w24bot=3w24bot
AccompanyBot=AccompanyBot
AdAuth=AdAuth
Adbeat bot=Adbeat bot
AdBot=AdBot
AddThis.com=AddThis.com
Adform crawler=Adform crawler
ADmantX crawler=ADmantX crawler
AdminLabs=AdminLabs
Adsbot=Adsbot
AdsBot-Google=AdsBot-Google
AdsBot-Google-Mobile=AdsBot-Google-Mobile
AdsrvrBot=AdsrvrBot
adstxt.com crawler=adstxt.com crawler
AdsTxtCrawler=AdsTxtCrawler
adstxtlab.com Crawler=adstxtlab.com Crawler
AffiliateLabz crawler=AffiliateLabz crawler
agentslug=agentslug
AhrefsBot=AhrefsBot
aiHitBot=aiHitBot
Akamai crawler=Akamai crawler
Alendo Bot=Alendo Bot
Alertra=Alertra
Alexabot=Alexabot
AlphaSeoBot=AlphaSeoBot
alyze.info bot=alyze.info bot
Amazon bot=Amazon bot
Amazon CloudFront=Amazon CloudFront
AmazonAdBot=AmazonAdBot
Amazonbot=Amazonbot
Anturis Agent=Anturis Agent
Apercite bot=Apercite bot
APIs-Google=APIs-Google
AppBeat=AppBeat
Applebot=Applebot
AppNexus crawler=AppNexus crawler
Aranea=Aranea
archive.org_bot=archive.org_bot
Archive.St crawler=Archive.St crawler
ArchiveBox=ArchiveBox
ArchiveTeam crawler=ArchiveTeam crawler
AspiegelBot=AspiegelBot
Aspire Web Crawler=Aspire Web Crawler
Atlas Comm crawler=Atlas Comm crawler
Atomseobot=Atomseobot
Automation Anywhere internal monitor=Automation Anywhere internal monitor
Awario crawler=Awario crawler
awin.com crawler=awin.com crawler
aylienbot=aylienbot
backlink-check.de=backlink-check.de
BacklinkCrawler=BacklinkCrawler
BackupLand crawler=BackupLand crawler
Bad-Neighborhood=Bad-Neighborhood
BadooBot=BadooBot
Baiduspider (bad_bot)=Baiduspider
BananaBot=BananaBot
Barkrowler=Barkrowler
bbot=bbot
BDCbot=BDCbot
bias-goggles crawler=bias-goggles crawler
bidswitchbot=bidswitchbot
BigInfoLabs crawler=BigInfoLabs crawler
bingbot=bingbot
BingPreview=BingPreview
BirilBot=BirilBot
bitlybot=bitlybot
Bitrix crawler=Bitrix crawler
bl.uk_lddc_bot=bl.uk_lddc_bot
Blackboard crawler=Blackboard crawler
BLEXBot=BLEXBot
blogger.com=blogger.com
Blogtrottr=Blogtrottr
Blue Prism Internal Monitor=Blue Prism Internal Monitor
bne.es crawler=bne.es crawler
bnf.fr_bot=bnf.fr_bot
BoardReader crawler=BoardReader crawler
BomboraBot=BomboraBot
BorneoBot=BorneoBot
BrandProtect bot=BrandProtect bot
brands-bot=brands-bot
brands-bot-logo=brands-bot-logo
BrandVerity=BrandVerity
Brandwatch crawler=Brandwatch crawler
BrokenLinkCheck.com=BrokenLinkCheck.com
Browsershots=Browsershots
BrowserSpyBot=BrowserSpyBot
BUbiNG=BUbiNG
BublupBot=BublupBot
BuiltWith crawler=BuiltWith crawler
bumb.ly crawler=bumb.ly crawler
Buzzbot=Buzzbot
ByteDance crawler=ByteDance crawler
Bytespider (bad_bot)=Bytespider
CastFeedValidator=CastFeedValidator
CATExplorador=CATExplorador
CCBot=CCBot
censys.io crawler=censys.io crawler
centurybot9=centurybot9
Check-Host=Check-Host
CheckMarkNetwork=CheckMarkNetwork
chimebot=chimebot
Chrome-Lighthouse=Chrome-Lighthouse
Cincraw=Cincraw
Cirkwi crawler=Cirkwi crawler
CISPA crawler=CISPA crawler
Clarabot=Clarabot
CLcrawler=CLcrawler
Cliqzbot=Cliqzbot
Closure Compiler Service=Closure Compiler Service
Cloud mapping=Cloud mapping
CloudFlare crawler=CloudFlare crawler
Cloudinary=Cloudinary
CMS Crawler=CMS Crawler
coccocbot=coccocbot
Cocolyze bot=Cocolyze bot
CommaFeed=CommaFeed
COMSYS crawler=COMSYS crawler
ContentKing=ContentKing
Corax=Corax
CouponWCode Bot=CouponWCode Bot
Crawler4j=Crawler4j
CrawlForMe=CrawlForMe
Crawlson=Crawlson
CronDroid=CronDroid
Cronjob.de bot=Cronjob.de bot
cron-job.org=cron-job.org
Cronless=Cronless
CrowdTanglebot=CrowdTanglebot
CSIRT.CZ crawler=CSIRT.CZ crawler
Cula=Cula
Cydral bot=Cydral bot
DareBoost=DareBoost
dark_NeXus_Qbot=dark_NeXus_Qbot
datagnionbot=datagnionbot
Datanyze=Datanyze
Dataprovider Spider=Dataprovider Spider
DataXu=DataXu
Daumoa=Daumoa
Dead Link Checker=Dead Link Checker
deadlinkchecker=deadlinkchecker
deepcrawl=deepcrawl
demandbase-bot=demandbase-bot
Deskyobot=Deskyobot
DF Bot=DF Bot
Diffbot=Diffbot
DingTalkBot=DingTalkBot
Discordbot=Discordbot
Disqus=Disqus
dlvr.it=dlvr.it
DNS-Tools Header-Analyzer=DNS-Tools Header-Analyzer
DocBase Crawler=DocBase Crawler
Docoloc=Docoloc
Domains Project bot=Domains Project bot
domainsbot=domainsbot
DomainSigmaCrawler=DomainSigmaCrawler
DomainStatsBot=DomainStatsBot
DomCop Bot=DomCop Bot
DotBot=DotBot
DownloaderChrome=DownloaderChrome
downnotifier.com monitoring=downnotifier.com monitoring
DowntimeDetector=DowntimeDetector
draw.io=draw.io
DuckDuckBot=DuckDuckBot
e.ventures crawler=e.ventures crawler
EasyBib crawler=EasyBib crawler
EasyCron=EasyCron
Easy-Thumb=Easy-Thumb
eBot=eBot
EchoboxBot=EchoboxBot
Elefent=Elefent
Elisabot=Elisabot
Embedly=Embedly
EPFL cravler=EPFL cravler
evc-batch=evc-batch
Exabot=Exabot
Experian crawler=Experian crawler
ExpertSearchSpider=ExpertSearchSpider
FacebookBot=FacebookBot
Fake bot=Fake bot
FAST Enterprise Crawler=FAST Enterprise Crawler
FastCrawler=FastCrawler
FAST-WebCrawler=FAST-WebCrawler
Faveeo=Faveeo
faviconkit=faviconkit
FCCN crawler=FCCN crawler
Feedbin=Feedbin
FeedBunch=FeedBunch
FeedBurner=FeedBurner
feeder.co=feeder.co
Feedfetcher-Google=Feedfetcher-Google
Feedly=Feedly
Feedpresso crawler=Feedpresso crawler
Feedspotbot=Feedspotbot
FeedViewer=FeedViewer
Feedwind=Feedwind
FemtosearchBot=FemtosearchBot
Filestack=Filestack
filibot=filibot
FindITAnswersbot=FindITAnswersbot
FirmoGraph=FirmoGraph
FlipboardBot=FlipboardBot
Foregenix crawler=Foregenix crawler
FreeWebMonitoring SiteChecker=FreeWebMonitoring SiteChecker
Freshworks bot=Freshworks bot
FullStoryBot=FullStoryBot
Fuze_Bot=Fuze_Bot
g2reader-bot=g2reader-bot
GAChecker=GAChecker
GDNP crawler=GDNP crawler
GetintentCrawler=GetintentCrawler
GetLinkInfo.com=GetLinkInfo.com
Ghost Inspector=Ghost Inspector
Gigabot=Gigabot
github-camo=github-camo
Gluten Free Crawler=Gluten Free Crawler
gocrawl=gocrawl
Google AdSense=Google AdSense
Google AMP crawler=Google AMP crawler
Google Apps Script=Google Apps Script
Google AppsViewer=Google AppsViewer
Google Cloud AI=Google Cloud AI
Google Cloud Monitoring=Google Cloud Monitoring
Google Favicon=Google Favicon
Google Page Speed Insights=Google Page Speed Insights
Google Read Aloud=Google Read Aloud
Google Web Preview=Google Web Preview
Google-Ads=Google-Ads
Google-AdWords=Google-AdWords
Google-Adwords-Instant=Google-Adwords-Instant
Googlebot AdsBot Mobile=AdsBot-Google-Mobile
Googlebot AdSense=Mediapartners-Google
Googlebot Image=Googlebot-Image
Googlebot Mobile=Googlebot-Mobile
Googlebot Snippet=Googlebot snippet
Googlebot Video=Googlebot-Video
Googlebot Web Preview=Google Web Preview
GoogleBot=GoogleBot
Googlebot=Googlebot
Googlebot-Image=Googlebot-Image
Googlebot-Mobile=Googlebot-Mobile
Googlebot-News=Googlebot-News
Googlebot-Video=Googlebot-Video
GoogleDocs=GoogleDocs
Google-PhysicalWeb=Google-PhysicalWeb
Google-Shopping=Google-Shopping
GoogleSites=GoogleSites
Google-Site-Verification=Google-Site-Verification
Google-Structured-Data-Testing-Tool=Google-Structured-Data-Testing-Tool
Google-Test=Google-Test
google-xrawler=google-xrawler
Google-Youtube-Links=Google-Youtube-Links
Gosign crawler=Gosign crawler
GotSiteMonitor=GotSiteMonitor
Gowikibot=Gowikibot
Grabify.link=Grabify.link
Grammarly=Grammarly
GrapeshotCrawler=GrapeshotCrawler
Grobbot=Grobbot
GroupHigh=GroupHigh
GTmetrix=GTmetrix
GumGum Bot=GumGum Bot
HappyApps crawler=HappyApps crawler
Hatena=Hatena
hCardValidator=hCardValidator
HetrixTools crawler=HetrixTools crawler
Heurekabot=Heurekabot
Hexometer.com crawler=Hexometer.com crawler
historious=historious
hledejLevne.cz=hledejLevne.cz
Hlidam.to robot=Hlidam.to robot
HostTracker=HostTracker
Http2Bot=Http2Bot
HuaweiWebCatBot=HuaweiWebCatBot
HubSpot Crawler=HubSpot Crawler
Hypefactors crawler=Hypefactors crawler
HyperZbozi.cz Feeder=HyperZbozi.cz Feeder
HypeStat=HypeStat
IAS crawler=IAS crawler
Iframely=Iframely
iGooglePortal=iGooglePortal
ImageFetcher=ImageFetcher
Impact Radius Bot=Impact Radius Bot
ImplisenseBot=ImplisenseBot
IndeedBot=IndeedBot
Info bot=Info bot
Infosecbot=Infosecbot
InternetSeer=InternetSeer
internetVista monitor=internetVista monitor
internetwache.org crawler=internetwache.org crawler
iodc crawler=iodc crawler
IPIP.NET crawler=IPIP.NET crawler
ips-agent=ips-agent
ipv6-test.com validator=ipv6-test.com validator
iqdb=iqdb
Irokez.cz monitoring=Irokez.cz monitoring
IstellaBot=IstellaBot
isUp.li Website Monitoring=isUp.li Website Monitoring
JAHHO crawler=JAHHO crawler
JamesBOT=JamesBOT
Jeffrey Exif Viewer=Jeffrey Exif Viewer
JobboerseBot=JobboerseBot
JobKereso=JobKereso
Jooblebot=Jooblebot
kakao crawler=kakao crawler
kansalliskirjasto.fi crawler=kansalliskirjasto.fi crawler
Kaspersky Lab bot=Kaspersky Lab bot
kazbtbot=kazbtbot
KD Bot=KD Bot
KeepRight OpenStreetMap Checker=KeepRight OpenStreetMap Checker
Keybot Translation=Keybot Translation
KeyCDN bot=KeyCDN bot
Kinza (bad_bot)=Kinza
KOCMOHABT bot=KOCMOHABT bot
KomodiaBot=KomodiaBot
Law Insider bot=Law Insider bot
leady.com crawler=leady.com crawler
LetsearchBot=LetsearchBot
letsencrypt bot=letsencrypt bot
libwww-perl=libwww-perl
library.unt.edu bot=library.unt.edu bot
Licorne crawler=Licorne crawler
LieBaoFast (bad_bot)=LieBaoFast
LightspeedSystems Crawler=LightspeedSystems Crawler
Linespider=Linespider
Linguee Bot=Linguee Bot
Link Valet Online=Link Valet Online
link_thumbnailer=link_thumbnailer
LinkAlarm=LinkAlarm
linkdexbot=linkdexbot
LinkedInBot=LinkedInBot
linkfluence=linkfluence
linkgraph.io crawler=linkgraph.io crawler
LinkpadBot=LinkpadBot
linkpeek=linkpeek
LinkTiger=LinkTiger
LinqiaBot=LinqiaBot
LinqTo.me=LinqTo.me
LivelapBot=LivelapBot
ltx71=ltx71
LumtelBot=LumtelBot
Mail.Ru bot=Mail.Ru bot
Mailkit crawler=Mailkit crawler
Mappy=Mappy
Marfeel crawler=Marfeel crawler
MarketingMiner Bot=MarketingMiner Bot
MarketMuseBot=MarketMuseBot
masscan=masscan
MauiBot=MauiBot
MAZBot=MAZBot
Mb2345Browser (bad_bot)=Mb2345Browser
MB-SiteCrawler=MB-SiteCrawler
Mediapartners=Mediapartners
Mediapartners-Google=Mediapartners-Google
Mediatoolkitbot=Mediatoolkitbot
Mediumbot=Mediumbot
MegaIndex=MegaIndex
MergadoBot=MergadoBot
MetaInspector=MetaInspector
MetaJobBot=MetaJobBot
MicroMessenger (bad_bot)=MicroMessenger
Minds crawler=Minds crawler
mindUpBot=mindUpBot
Miniature.io=Miniature.io
Miniflux=Miniflux
Mixnode=Mixnode
MixrankBot=MixrankBot
MJ12bot=MJ12bot
Moatbot=Moatbot
MojeekBot=MojeekBot
Monibot=Monibot
monitis=monitis
monitorbacklinks.com bot=monitorbacklinks.com bot
montastic-monitor=montastic-monitor
MonTools.Com=MonTools.Com
Moreover=Moreover
msnbot=msnbot
msrbot=msrbot
MuckRack=MuckRack
NameFo=NameFo
Naver Scrap=Naver Scrap
Naver scrapbook=Naver scrapbook
NaverBot=NaverBot
nbertaupete95=nbertaupete95
netarkivet.dk crawler=netarkivet.dk crawler
Netcraft crawler=Netcraft crawler
netEstate Crawler=netEstate Crawler
Neticle Crawler=Neticle Crawler
Netpeak bot=Netpeak bot
NetSystemsResearch crawler=NetSystemsResearch crawler
Netvibes crawler=Netvibes crawler
netzzappen crawler=netzzappen crawler
Neustar WPM=Neustar WPM
Nextcloud Crawler=Nextcloud Crawler
Nimbostratus-Bot=Nimbostratus-Bot
NinjaBot=NinjaBot
niocBot=niocBot
NiueBot=NiueBot
NIXStatsbot=NIXStatsbot
Nmap=Nmap
Nodemeter=Nodemeter
NodePing=NodePing
nomore404.com robot=nomore404.com robot
NortheasternSysNetBot=NortheasternSysNetBot
Nuzzel=Nuzzel
oasis.go.kr crawler=oasis.go.kr crawler
oBot=oBot
OdklBot=OdklBot
OgScrper=OgScrper
OhDear crawler=OhDear crawler
omgilibot=omgilibot
Onespot crawler=Onespot crawler
Online Domain Tools=Online Domain Tools
Opengraph.io bot=Opengraph.io bot
OpenGraphCheck=OpenGraphCheck
OpenindexSpider=OpenindexSpider
Openstat=Openstat
OrgProbe=OrgProbe
OSZKbot=OSZKbot
Owlin=Owlin
p40Bot=p40Bot
PageModified Crawler=PageModified Crawler
PagePeeker=PagePeeker
Pagespeed=Pagespeed
Panopta=Panopta
Panscient crawler=Panscient crawler
PaperLiBot=PaperLiBot
PartyflockLinkChecker=PartyflockLinkChecker
PDFDriveCrawler=PDFDriveCrawler
Pearltrees=Pearltrees
People Data Labs crawler=People Data Labs crawler
PetalBot=PetalBot
PhantomJsCloud.com=PhantomJsCloud.com
Photon=Photon
phpservermon=phpservermon
pilicanbot=pilicanbot
PimEyes crawler=PimEyes crawler
Pingability.com bot=Pingability.com bot
PingAdmin.Ru=PingAdmin.Ru
Pingdom.com bot=Pingdom.com bot
Pinterest bot=Pinterest bot
PLAZOO bot=PLAZOO bot
Plukkie=Plukkie
PlurkBot=PlurkBot
Pocket bot=Pocket bot
polaris botnet=polaris botnet
Port Monitor check service=Port Monitor check service
PowerMapper Crawler=PowerMapper Crawler
pr-cy.ru Bot=pr-cy.ru Bot
Premsgo.fr bot=Premsgo.fr bot
Prerender.cloud=Prerender.cloud
Prerender=Prerender
PrintFriendly.com=PrintFriendly.com
PritTorrent=PritTorrent
PrivacyScore.org crawler=PrivacyScore.org crawler
Prlog=Prlog
Project 25499=Project 25499
Promotion_Tools=Promotion_Tools
Protopage=Protopage
proximic=proximic
psbot=psbot
PubMatic Crawler=PubMatic Crawler
PulnoBot=PulnoBot
QuoraBot=QuoraBot
Qwantify=Qwantify
Rambler bot=Rambler bot
RankurBot=RankurBot
RavenCrawler=RavenCrawler
REDbot=REDbot
redditbot=redditbot
Re-re crawler=Re-re crawler
Rival IQ=Rival IQ
Robots_Tester=Robots_Tester
RobotsChecker=RobotsChecker
rogerbot=rogerbot
Roistat Bot=Roistat Bot
ROR Sitemap Generator=ROR Sitemap Generator
RSiteAuditor=RSiteAuditor
RSSingBot=RSSingBot
RSSMicro.com RSS=RSSMicro.com RSS
RUB crawler=RUB crawler
RukiCrawler=RukiCrawler
Runet-Research-Crawler=Runet-Research-Crawler
RyowlEngine=RyowlEngine
Ryte Bot=Ryte Bot
s4a=s4a
SafeDNSBot=SafeDNSBot
SBSearch=SBSearch
SCFCrawler=SCFCrawler
ScopeContent bot=ScopeContent bot
Scrapy=Scrapy
screeenly-bot=screeenly-bot
SE Ranking bot=SE Ranking bot
Seeker=Seeker
Seekport Crawler=Seekport Crawler
SeeWithKids bot=SeeWithKids bot
SemanticJuice crawler=SemanticJuice crawler
SemanticScholarBot=SemanticScholarBot
semantic-visions.com crawler=semantic-visions.com crawler
SEMrushBot=SEMrushBot
SemtixBot=SemtixBot
SentiBot=SentiBot
Seo Servis=Seo Servis
Seobility=Seobility
SEOdiver=SEOdiver
SEOkicks-Robot=SEOkicks-Robot
SEOlizer=SEOlizer
SeopultContentAnalyzer=SeopultContentAnalyzer
seoscanners.net bot=seoscanners.net bot
SeoSiteCheckup=SeoSiteCheckup
SerendeputyBot=SerendeputyBot
serpstatbot=serpstatbot
SERPtimizerBot=SERPtimizerBot
ServiceUptime.robot=ServiceUptime.robot
SetCronJob=SetCronJob
Seznam Zbozi.cz=Seznam Zbozi.cz
SeznamBot=SeznamBot
Sideqik crawler=Sideqik crawler
Sidetrade crawler=Sidetrade crawler
SimplyFast.info=SimplyFast.info
Sistrix crawler=Sistrix crawler
SISTRIX Optimizer=SISTRIX Optimizer
Site24x7=Site24x7
SiteCheckerBot=SiteCheckerBot
Siteimprove.com bot=Siteimprove.com bot
SiteTruth.com=SiteTruth.com
SiteUptime.com=SiteUptime.com
SJUUPBot=SJUUPBot
SKB Kontur bot=SKB Kontur bot
SkypeUriPreview=SkypeUriPreview
Slackbot=Slackbot
SMTBot=SMTBot
snapchat-proxy=snapchat-proxy
SnapSearch=SnapSearch
sniptracker=sniptracker
Sogou (bad_bot)=Sogou
sogou spider=sogou spider
SpazioDati bot=SpazioDati bot
Specificfeeds=Specificfeeds
spider=spider
SpiderLing=SpiderLing
Splash=Splash
SpotiboBot=SpotiboBot
SpuhexBot=SpuhexBot
SpyOnWeb=SpyOnWeb
SSL Labs=SSL Labs
sstbot=sstbot
startmebot=startmebot
statdom.ru=statdom.ru
Statically crawler=Statically crawler
statista.com crawler=statista.com crawler
StatOnlineRuBot=StatOnlineRuBot
StatusCake=StatusCake
Statusoid=Statusoid
StatvooBot=StatvooBot
Steeler=Steeler
Stephan Kopp bot=Stephan Kopp bot
Streamline3Bot=Streamline3Bot
Structured Data Markup Helper=Structured Data Markup Helper
Sucuri bot=Sucuri bot
Super Monitoring=Super Monitoring
SurdotlyBot=SurdotlyBot
SvetaBot=SvetaBot
Synthesio Crawler=Synthesio Crawler
Taringa=Taringa
TBI Project bot=TBI Project bot
TelegramBot=TelegramBot
Testomatobot=Testomatobot
TestURI crawler=TestURI crawler
TextRazor crawler=TextRazor crawler
The Knowledge AI=The Knowledge AI
theoldreader.com=theoldreader.com
thumbshots-de-Bot=thumbshots-de-Bot
TinEye=TinEye
TMM Crawler=TMM Crawler
TomBot=TomBot
Tools4noobs.com Spider=Tools4noobs.com Spider
TprAdsTxtCrawler=TprAdsTxtCrawler
Traackr.com=Traackr.com
tracemyfile=tracemyfile
trendictionbot=trendictionbot
TrendsmapResolver=TrendsmapResolver
TripleCheckerRobot=TripleCheckerRobot
TU Braunschweig crawler=TU Braunschweig crawler
TurnitinBot=TurnitinBot
TweetmemeBot=TweetmemeBot
Twingly Recon=Twingly Recon
Twitterbot=Twitterbot
UASlinkChecker=UASlinkChecker
uMBot=uMBot
umich.edu crawler=umich.edu crawler
Unshorten.It=Unshorten.It
updown.io bot=updown.io bot
Uptimebot=Uptimebot
UptimeRobot=UptimeRobot
Uptimia=Uptimia
uptm.io=uptm.io
Urlcheckr=Urlcheckr
URLitor.com=URLitor.com
Vagabondo=Vagabondo
vebidoobot=vebidoobot
VelenPublicWebCrawler=VelenPublicWebCrawler
Veoozbot=Veoozbot
VerticalLeap bot=VerticalLeap bot
Virusdie crawler=Virusdie crawler
virustotal=virustotal
vkShare=vkShare
VSAgent=VSAgent
vuhuvBot=vuhuvBot
W3C_CSS_Validator=W3C_CSS_Validator
W3C_I18n-Checker=W3C_I18n-Checker
W3C_Unicorn=W3C_Unicorn
W3C_Validator=W3C_Validator
W3C-checklink=W3C-checklink
W3C-mobileOK=W3C-mobileOK
w3dt.net crawler=w3dt.net crawler
Wappalyzer=Wappalyzer
WatchMouse=WatchMouse
Watson Explorer=Watson Explorer
Web Crawler=Web Crawler
Web Manifest Validator=Web Manifest Validator
webarchiv.cz=webarchiv.cz
WebBot=WebBot
WebCookies=WebCookies
WebCrawler=WebCrawler
Webcron.org crawler=Webcron.org crawler
WebDataStats=WebDataStats
webdepozit.sk crawler=webdepozit.sk crawler
WebGazer=WebGazer
weborama crawler=weborama crawler
Webshot=Webshot
Website-audit.be Crawler=Website-audit.be Crawler
websitepulse bot=websitepulse bot
WebSniffer=WebSniffer
WebTechSurvey bot=WebTechSurvey bot
WebThumb=WebThumb
WEDOS OnLine monitoring=WEDOS OnLine monitoring
WellknownWatcher=WellknownWatcher
WGETbot=WGETbot
WhatCMS.org bot=WhatCMS.org bot
WhoAPI=WhoAPI
Wikipedia crawler=Wikipedia crawler
wmtips.com=wmtips.com
woobot=woobot
WordPress.com bots=WordPress.com bots
WormlyBot=WormlyBot
WPSec=WPSec
WURFL ImageEngine=WURFL ImageEngine
XML Schema Validator=XML Schema Validator
XML Sitemaps Generator=XML Sitemaps Generator
XmlSitemapGenerator=XmlSitemapGenerator
XoviBot=XoviBot
XTC BOTNET=XTC BOTNET
yacybot=yacybot
Yahoo!=Yahoo!
Yandex.Metrica=Yandex.Metrica
Yandex.Webmaster=Yandex.Webmaster
YandexBot=YandexBot
Yellowbrandprotectionbot=Yellowbrandprotectionbot
YellowLabTools=YellowLabTools
YFF35=YFF35
YioopBot=YioopBot
YisouSpider=YisouSpider
YP crawler=YP crawler
zh_CN (bad_bot)=zh_CN
zh-CN (bad_bot)=zh-CN
Zombiebot=Zombiebot
ZoomBot=ZoomBot
ZoomInfo bot=ZoomInfo bot
Kinza (bad_bot)=Kinza
LieBaoFast (bad_bot)=LieBaoFast
Mb2345Browser (bad_bot)=Mb2345Browser
MicroMessenger (bad_bot)=MicroMessenger
Sogou (bad_bot)=Sogou
zh-CN (bad_bot)=zh-CN
zh_CN (bad_bot)=zh_CN



The following lists are included within this post for historical completeness.

Fresh Installs of UBB.threads 7.7.4 are pre-populated with this list.
The 2020-01-14 List:
Code
007ac9=007ac9
13TABS=13TABS
1stwarning bot=1stwarning bot
28logsSpider=28logsSpider
2bone=2bone
2GDPR bot=2GDPR bot
2gisbot=2gisbot
2ip.ru=2ip.ru
360Spider=360Spider
3w24bot=3w24bot
A6-Indexer=A6-Indexer
Aboundex=Aboundex
acapbot=acapbot
acoonbot=acoonbot
AdAuth=AdAuth
Adbeat bot=Adbeat bot
adbeat_bot=adbeat_bot
AddSearchBot=AddSearchBot
AddThis.com=AddThis.com
AddThis=AddThis
Adform crawler=Adform crawler
Adidxbot=Adidxbot
ADmantX crawler=ADmantX crawler
ADmantX=ADmantX
AdminLabs=AdminLabs
Adolab bot=Adolab bot
AdsBot-Google=AdsBot-Google
AdsBot-Google-Mobile=AdsBot-Google-Mobile
adscanner=adscanner
AdsrvrBot=AdsrvrBot
adstxt.com crawler=adstxt.com crawler
AdsTxtCrawler=AdsTxtCrawler
adstxtlab.com Crawler=adstxtlab.com Crawler
AdvBot=AdvBot
agentslug=agentslug
Ahrefs=Ahrefs
aiHitBot=aiHitBot
AISearchBot=AISearchBot
Akamai crawler=Akamai crawler
Alertra=Alertra
Alexabot=Alexabot
Alignabot=Alignabot
AlphaBot=AlphaBot
AlphaSeoBot=AlphaSeoBot
alyze.info=alyze.info
Amazon bot=Amazon bot
Amazon CloudFront=Amazon CloudFront
AmazonAdBot=AmazonAdBot
AndersPinkBot=AndersPinkBot
AntBot=AntBot
antibot=antibot
Anturis Agent=Anturis Agent
AnyEvent=AnyEvent
Apercite bot=Apercite bot
Apercite=Apercite
APIs-Google=APIs-Google
AppBeat=AppBeat
AppEngine-Google=AppEngine-Google
AppInsights=AppInsights
Applebot=Applebot
AppNexus crawler=AppNexus crawler
arabot=arabot
arachnode.net=arachnode.net
Aranea=Aranea
archive.org_bot=archive.org_bot
Archive.St crawler=Archive.St crawler
ArchiveBot=ArchiveBot
ArchiveTeam crawler=ArchiveTeam crawler
Awario crawler=Awario crawler
awesomecrawler=awesomecrawler
axios=axios
aylienbot=aylienbot
B2B Bot=B2B Bot
backlink-check.de=backlink-check.de
BacklinkCrawler=BacklinkCrawler
BackupLand crawler=BackupLand crawler
Bad-Neighborhood=Bad-Neighborhood
Baiduspider=Baiduspider
Baidu-YunGuanCe=Baidu-YunGuanCe
Barkrowler=Barkrowler
BazQux=BazQux
BDCbot=BDCbot
BehloolBot=BehloolBot
betaBot=betaBot
bidswitchbot=bidswitchbot
BigInfoLabs crawler=BigInfoLabs crawler
BIGLOTRON=BIGLOTRON
bingbot=bingbot
BingPreview=BingPreview
binlar=binlar
BirilBot=BirilBot
bitlybot=bitlybot
bl.uk_lddc_bot=bl.uk_lddc_bot
Blackboard crawler=Blackboard crawler
Blackboard=Blackboard
BLEXBot=BLEXBot
blogmuraBot=blogmuraBot
Blogtrottr=Blogtrottr
BLP_bbot=BLP_bbot
bne.es crawler=bne.es crawler
bnf.fr_bot=bnf.fr_bot
BoardReader crawler=BoardReader crawler
BomboraBot=BomboraBot
Bot.AraTurka.com=Bot.AraTurka.com
botify=botify
bot-pge.chlooe.com=bot-pge.chlooe.com
BoxcarBot=BoxcarBot
brainobot=brainobot
BrandProtect bot=BrandProtect bot
BrandVerity=BrandVerity
Brandwatch crawler=Brandwatch crawler
BrokenLinkCheck.com=BrokenLinkCheck.com
Browsershots=Browsershots
BrowserSpyBot=BrowserSpyBot
BTWebClient=BTWebClient
BUbiNG=BUbiNG
BublupBot=BublupBot
Buck=Buck
BuiltWith=BuiltWith
bumb.ly crawler=bumb.ly crawler
Burf.co=Burf.co
buzzbot=buzzbot
ByteDance crawler=ByteDance crawler
Bytespider=Bytespider
Caliperbot=Caliperbot
CapsuleChecker=CapsuleChecker
careerbot=careerbot
CastFeedValidator=CastFeedValidator
CATExplorador=CATExplorador
CC Metadata Scaper=CC Metadata Scaper
CCBot=CCBot
centurybot9=centurybot9
changedetection=changedetection
check_http=check_http
Check-Host=Check-Host
CheckMarkNetwork=CheckMarkNetwork
Chirp=Chirp
Chrome-Lighthouse=Chrome-Lighthouse
citeseerxbot=citeseerxbot
CityGrid crawler=CityGrid crawler
Clarabot=Clarabot
CLcrawler=CLcrawler
Clickagy=Clickagy
Cliqzbot=Cliqzbot
Closure Compiler Service=Closure Compiler Service
Cloud mapping=Cloud mapping
CloudFlare crawler=CloudFlare crawler
CloudFlare-AlwaysOnline=CloudFlare-AlwaysOnline
Cloudinary=Cloudinary
CMS Crawler=CMS Crawler
coccoc=coccoc
Cocolyze bot=Cocolyze bot
[email protected][email protected]
CommaFeed=CommaFeed
Companybook-Crawler=Companybook-Crawler
COMSYS crawler=COMSYS crawler
content crawler spider=content crawler spider
ContentKing=ContentKing
ContextAd Bot=ContextAd Bot
contxbot=contxbot
convera=convera
CopperEgg=CopperEgg
Corax=Corax
CouponWCode Bot=CouponWCode Bot
Coveobot=Coveobot
crawler4j=crawler4j
CrawlForMe=CrawlForMe
CronDroid=CronDroid
Cronjob.de bot=Cronjob.de bot
cron-job.org=cron-job.org
Cronless=Cronless
CrowdTanglebot=CrowdTanglebot
CrunchBot=CrunchBot
CrystalSemanticsBot=CrystalSemanticsBot
Cula=Cula
cXensebot=cXensebot
CyberPatrol=CyberPatrol
DaaS.sh crawler=DaaS.sh crawler
DareBoost=DareBoost
Datafeedwatch=Datafeedwatch
datagnionbot=datagnionbot
Datanyze=Datanyze
Dataprovider Spider=Dataprovider Spider
Dataprovider.com=Dataprovider.com
DataXu=DataXu
Daum=Daum
Daumoa=Daumoa
DaveCrawler=DaveCrawler
DAWINCI ANTIPLAG=DAWINCI ANTIPLAG
dcrawl=dcrawl
ddline.cn crawler=ddline.cn crawler
Dead Link Checker=Dead Link Checker
deadlinkchecker=deadlinkchecker
deepcrawl=deepcrawl
demandbase-bot=demandbase-bot
Deskyobot=Deskyobot
DeuSu=DeuSu
DF Bot=DF Bot
Diffbot=Diffbot
Digg Deeper=Digg Deeper
Digincore bot=Digincore bot
DIGMATO.com crawler=DIGMATO.com crawler
discobot=discobot
Discordbot=Discordbot
Disqus=Disqus
dlvr.it=dlvr.it
DNS-Tools Header-Analyzer=DNS-Tools Header-Analyzer
DnyzBot=DnyzBot
Docoloc=Docoloc
Domain Re-Animator Bot=Domain Re-Animator Bot
domaincrawler=domaincrawler
domainsbot=domainsbot
DomainsDB.net MetaCrawler=DomainsDB.net MetaCrawler
DomainSigmaCrawler=DomainSigmaCrawler
DomainStatsBot=DomainStatsBot
dotbot=dotbot
Dotcom-Monitor bot=Dotcom-Monitor bot
DownloaderChrome=DownloaderChrome
downnotifier.com monitoring=downnotifier.com monitoring
DowntimeDetector=DowntimeDetector
draw.io=draw.io
drupact=drupact
DuckDuckBot=DuckDuckBot
DuckDuckGo-Favicons-Bot=DuckDuckGo-Favicons-Bot
Dun&BradstreetBot=Dun&BradstreetBot
e.ventures crawler=e.ventures crawler
EasyBib crawler=EasyBib crawler
EasyCron=EasyCron
Easy-Thumb=Easy-Thumb
ec2linkfinder=ec2linkfinder
EchoboxBot=EchoboxBot
edisterbot=edisterbot
electricmonk=electricmonk
Elefent=Elefent
elisabot=elisabot
Embedly=Embedly
EnliteAI crawler=EnliteAI crawler
EPFL cravler=EPFL cravler
epicbot=epicbot
eright=eright
europarchive.org=europarchive.org
evc-batch=evc-batch
Everyonedomainsbot=Everyonedomainsbot
EveryoneSocialBot=EveryoneSocialBot
exabot=exabot
Experian crawler=Experian crawler
Experibot=Experibot
ExtLinksBot=ExtLinksBot
EZID=EZID
ezooms=ezooms
FacebookBot=FacebookBot
facebookexternalhit=facebookexternalhit
Facebot=Facebot
Fake bot=Fake bot
FAST Enterprise Crawler=FAST Enterprise Crawler
FastCrawler=FastCrawler
FAST-WebCrawler=FAST-WebCrawler
Favicon=Favicon
faviconkit=faviconkit
FCCN crawler=FCCN crawler
fedoraplanet=fedoraplanet
Feedbin=Feedbin
FeedBunch=FeedBunch
FeedBurner=FeedBurner
feeder.co=feeder.co
Feedfetcher-Google=Feedfetcher-Google
Feedly=Feedly
Feedpresso crawler=Feedpresso crawler
Feedspot=Feedspot
Feedspotbot=Feedspotbot
FeedValidator=FeedValidator
FeedViewer=FeedViewer
Feedwind=Feedwind
FemtosearchBot=FemtosearchBot
Fetch=Fetch
Fever=Fever
Filestack=Filestack
filterdb.iss.netcrawler=filterdb.iss.netcrawler
FindITAnswersbot=FindITAnswersbot
findl.sk bot=findl.sk bot
findlink=findlink
findthatfile=findthatfile
findxbot=findxbot
FirmoGraph=FirmoGraph
Flamingo_SearchEngine=Flamingo_SearchEngine
FlipboardBot=FlipboardBot
FlipboardProxy=FlipboardProxy
fluffy=fluffy
fr-crawler=fr-crawler
freefind=freefind
FreeWebMonitoring SiteChecker=FreeWebMonitoring SiteChecker
Freshworks bot=Freshworks bot
Friendica=Friendica
Frontera=Frontera
fuelbot=fuelbot
Fuze_Bot=Fuze_Bot
Fyrebot=Fyrebot
g00g1e.net=g00g1e.net
G2 Web Services=G2 Web Services
g2reader-bot=g2reader-bot
GAChecker=GAChecker
GarlikCrawler=GarlikCrawler
GDNP crawler=GDNP crawler
Geek-Tools=Geek-Tools
Genieo Web filter=Genieo Web filter
Genieo=Genieo
GetLinkInfo.com=GetLinkInfo.com
Ghost Inspector=Ghost Inspector
Gigablast=Gigablast
Gigabot=Gigabot
GingerCrawler=GingerCrawler
github-camo=github-camo
Gluten Free Crawler=Gluten Free Crawler
gnam gnam spider=gnam gnam spider
GnowitNewsbot=GnowitNewsbot
gocrawl=gocrawl
Go-http-client=Go-http-client
Google AMP crawler=Google AMP crawler
Google AppsViewer=Google AppsViewer
Google Cloud AI=Google Cloud AI
Google Favicon=Google Favicon
Google Page Speed Insights=Google Page Speed Insights
Google Read Aloud=Google Read Aloud
Google Web Preview=Google Web Preview
Google-Ads=Google-Ads
Google-AdWords=Google-AdWords
Google-Adwords-Instant=Google-Adwords-Instant
Googlebot AdsBot Mobile=AdsBot-Google-Mobile
Googlebot AdSense=Mediapartners-Google
Googlebot Image=Googlebot-Image
Googlebot Mobile=Googlebot-Mobile
Googlebot Snippet=Googlebot snippet
Googlebot Video=Googlebot-Video
Googlebot Web Preview=Google Web Preview
Googlebot=Googlebot
Googlebot-Image=Googlebot-Image
Googlebot-Mobile=Googlebot-Mobile
Googlebot-News=Googlebot-News
Googlebot-Video=Googlebot-Video
GoogleDocs=GoogleDocs
Google-PhysicalWeb=Google-PhysicalWeb
GoogleSites=GoogleSites
Google-Site-Verification=Google-Site-Verification
Google-Structured-Data-Testing-Tool=Google-Structured-Data-Testing-Tool
Google-Test=Google-Test
google-xrawler=google-xrawler
Google-Youtube-Links=Google-Youtube-Links
Gosign crawler=Gosign crawler
GoSquared=GoSquared
GotSiteMonitor=GotSiteMonitor
Gowikibot=Gowikibot
Grammarly=Grammarly
GrapeshotCrawler=GrapeshotCrawler
Graydon crawler=Graydon crawler
Grobbot=Grobbot
GroupHigh=GroupHigh
grub.org=grub.org
gslfbot=gslfbot
GTmetrix=GTmetrix
GumGum Bot=GumGum Bot
HappyApps crawler=HappyApps crawler
Hatena=Hatena
hCardValidator=hCardValidator
HeadlessChrome=HeadlessChrome
HeartRailsBot=HeartRailsBot
heritrix=heritrix
HetrixTools crawler=HetrixTools crawler
Heurekabot=Heurekabot
Hexometer.com crawler=Hexometer.com crawler
historious=historious
hledejLevne.cz=hledejLevne.cz
Hlidam.to robot=Hlidam.to robot
hosterstats com=hosterstats com
HostTracker=HostTracker
htmlyse crawler=htmlyse crawler
http_get=http_get
httpunit=httpunit
HttpUrlConnection=HttpUrlConnection
HTTrack=HTTrack
HubSpot Crawler=HubSpot Crawler
HubSpot=HubSpot
Hypefactors crawler=Hypefactors crawler
HyperZbozi.cz Feeder=HyperZbozi.cz Feeder
HypeStat=HypeStat
ia_archiver=ia_archiver
IAB crawler=IAB crawler
IAS crawler=IAS crawler
ICBot=ICBot
ICC-Crawler=ICC-Crawler
ichiro=ichiro
iCjobs=iCjobs
Iframely=Iframely
iGooglePortal=iGooglePortal
ImageFetcher=ImageFetcher
ImplisenseBot=ImplisenseBot
imrbot=imrbot
IndeedBot=IndeedBot
Infegy=Infegy
InoopaBot=InoopaBot
integromedb=integromedb
intelium_bot=intelium_bot
InterfaxScanBot=InterfaxScanBot
InternetSeer=InternetSeer
internetVista monitor=internetVista monitor
internetwache.org crawler=internetwache.org crawler
iodc crawler=iodc crawler
IPIP.NET crawler=IPIP.NET crawler
ips-agent=ips-agent
ipv6-test.com validator=ipv6-test.com validator
ip-web-crawler.com=ip-web-crawler.com
iqdb=iqdb
Irokez.cz monitoring=Irokez.cz monitoring
Is it up=Is it up
isa=isa
iSec_Bot=iSec_Bot
iskanie=iskanie
ISOWQ=ISOWQ
IstellaBot=IstellaBot
isUp.li Website Monitoring=isUp.li Website Monitoring
it2media-domain-crawler=it2media-domain-crawler
IzABee Spider=IzABee Spider
JAHHO crawler=JAHHO crawler
James BOT=James BOT
JamesBOT=JamesBOT
Jamie's Spider=Jamie's Spider
Jeffrey Exif Viewer=Jeffrey Exif Viewer
Jetslide=Jetslide
Jetty=Jetty
JobboerseBot=JobboerseBot
JobKereso=JobKereso
Jooblebot=Jooblebot
jpg-newsbot=jpg-newsbot
Jugendschutzprogramm-Crawler=Jugendschutzprogramm-Crawler
jyxobot=jyxobot
K7MLWCBot=K7MLWCBot
kakao crawler=kakao crawler
kansalliskirjasto.fi crawler=kansalliskirjasto.fi crawler
Kaspersky Lab bot=Kaspersky Lab bot
kazbtbot=kazbtbot
KD Bot=KD Bot
KeepRight OpenStreetMap Checker=KeepRight OpenStreetMap Checker
Kemvibot=Kemvibot
Keybot Translation=Keybot Translation
KeyCDN bot=KeyCDN bot
keyword-hero bot=keyword-hero bot
KOCMOHABT bot=KOCMOHABT bot
KomodiaBot=KomodiaBot
KosmioBot=KosmioBot
Landau-Media-Spider=Landau-Media-Spider
Laserlikebot=Laserlikebot
lb-spider=lb-spider
LCC=LCC
Lead Inspector Crawler=Lead Inspector Crawler
leady.com crawler=leady.com crawler
Leikibot=Leikibot
LetsearchBot=LetsearchBot
letsencrypt bot=letsencrypt bot
libwww-perl=libwww-perl
Licorne crawler=Licorne crawler
LightspeedSystems Crawler=LightspeedSystems Crawler
Linespider=Linespider
Linguee Bot=Linguee Bot
Link Valet Online=Link Valet Online
link_thumbnailer=link_thumbnailer
LinkAlarm=LinkAlarm
linkapediabot=linkapediabot
LinkArchiver=LinkArchiver
linkdex=linkdex
linkdexbot=linkdexbot
LinkedInBot=LinkedInBot
linkfluence=linkfluence
LinkpadBot=LinkpadBot
linkpeek=linkpeek
LinkTiger=LinkTiger
LinqiaBot=LinqiaBot
lipperhey=lipperhey
LivelapBot=LivelapBot
looid.com crawler=looid.com crawler
lssbot=lssbot
lssrocketcrawler=lssrocketcrawler
ltx71=ltx71
Luminator-robots=Luminator-robots
LumtelBot=LumtelBot
LXRbot=LXRbot
MagiBot=MagiBot
magpie-crawler=magpie-crawler
Mail.Ru bot=Mail.Ru bot
Mail.RU_Bot=Mail.RU_Bot
Mailkit crawler=Mailkit crawler
Mappy=Mappy
mappydata=mappydata
MarketingMiner Bot=MarketingMiner Bot
MarketMuseBot=MarketMuseBot
masscan=masscan
Mastodon=Mastodon
MauiBot=MauiBot
MBCrawler=MBCrawler
MB-SiteCrawler=MB-SiteCrawler
Mediapartners=Mediapartners
Mediapartners-Google=Mediapartners-Google
Mediatoolkitbot=Mediatoolkitbot
Mediumbot=Mediumbot
MegaIndex=MegaIndex
MeltwaterNews=MeltwaterNews
memorybot=memorybot
Mergadobot=Mergadobot
MetaInspector=MetaInspector
MetaJobBot=MetaJobBot
MetaURI=MetaURI
Minds crawler=Minds crawler
mindUpBot=mindUpBot
Miniature.io=Miniature.io
Miniflux=Miniflux
Miralinks Robot=Miralinks Robot
Mixnode=Mixnode
MixnodeCache=MixnodeCache
MixrankBot=MixrankBot
MJ12bot=MJ12bot
mlbot=mlbot
moatbot=moatbot
MojeekBot=MojeekBot
Monibot=Monibot
monitis=monitis
Monitive Uptime Monitoring Bot=Monitive Uptime Monitoring Bot
monitorabot=monitorabot
monitorbacklinks.com bot=monitorbacklinks.com bot
montastic-monitor=montastic-monitor
MonTools.Com=MonTools.Com
MoodleBot=MoodleBot
Moreover=Moreover
msnbot=msnbot
msrbot=msrbot
MuckRack=MuckRack
Multiviewbot=Multiviewbot
Naver scrapbook=Naver scrapbook
NaverBot=NaverBot
nbertaupete95=nbertaupete95
NerdByNature.Bot=NerdByNature.Bot
nerdybot=nerdybot
netarkivet.dk crawler=netarkivet.dk crawler
Netcraft crawler=Netcraft crawler
NetcraftSurveyAgent=NetcraftSurveyAgent
netEstate Crawler=netEstate Crawler
netEstate NE Crawler=netEstate NE Crawler
Netpeak bot=Netpeak bot
netresearchserver=netresearchserver
NetSystemsResearch crawler=NetSystemsResearch crawler
Netvibes crawler=Netvibes crawler
Netvibes=Netvibes
netzzappen crawler=netzzappen crawler
newsharecounts=newsharecounts
newspaper=newspaper
NextCloud=NextCloud
niki-bot=niki-bot
Nimbostratus-Bot=Nimbostratus-Bot
NING=NING
NinjaBot=NinjaBot
NIXStatsbot=NIXStatsbot
Nmap Scripting Engine=Nmap Scripting Engine
Nmap=Nmap
Nodemeter=Nodemeter
NodePing=NodePing
northcutt.com crawler=northcutt.com crawler
NortheasternSysNetBot=NortheasternSysNetBot
NTENTbot=NTENTbot
nutch=nutch
Nuzzel=Nuzzel
oasis.go.kr crawler=oasis.go.kr crawler
oBot=oBot
Ocarinabot=Ocarinabot
OdklBot=OdklBot
OhDear crawler=OhDear crawler
okhttp=okhttp
omgili=omgili
omgilibot=omgilibot
OnCrawl=OnCrawl
Onespot crawler=Onespot crawler
Online Domain Tools=Online Domain Tools
online-webceo-bot=online-webceo-bot
OpenGraphCheck=OpenGraphCheck
OpenHoseBot=OpenHoseBot
openindexspider=openindexspider
Openstat=Openstat
OrangeBot=OrangeBot
OrgProbe=OrgProbe
OSZKbot=OSZKbot
outbrain=outbrain
OutclicksBot=OutclicksBot
Owlin=Owlin
page2rss=page2rss
PageModified Crawler=PageModified Crawler
PagePeeker=PagePeeker
Pagespeed=Pagespeed
Panopta=Panopta
Panscient crawler=Panscient crawler
panscient=panscient
PaperLiBot=PaperLiBot
ParaCrawl=ParaCrawl
PartyflockLinkChecker=PartyflockLinkChecker
Pcore-HTTP=Pcore-HTTP
PDFDriveCrawler=PDFDriveCrawler
Pearltrees=Pearltrees
PhantomJS=PhantomJS
PhantomJsCloud.com=PhantomJsCloud.com
Photon=Photon
phpcrawl=phpcrawl
phpservermon=phpservermon
pilicanbot=pilicanbot
Pingability.com bot=Pingability.com bot
PingAdmin.Ru=PingAdmin.Ru
Pingdom.com bot=Pingdom.com bot
pingdom=pingdom
Pinterest bot=Pinterest bot
pinterest.com.bot=pinterest.com.bot
PiplBot=PiplBot
PLAZOO bot=PLAZOO bot
PleskBot=PleskBot
Plukkie=Plukkie
PlurkBot=PlurkBot
Pocket bot=Pocket bot
PocketParser=PocketParser
Port Monitor check service=Port Monitor check service
postrank=postrank
PowerMapper Crawler=PowerMapper Crawler
pr-cy.ru Bot=pr-cy.ru Bot
PR-CY.RU=PR-CY.RU
Premsgo.fr bot=Premsgo.fr bot
Prerender.cloud=Prerender.cloud
Prerender=Prerender
presearch crawler=presearch crawler
Primalbot=Primalbot
PrintFriendly.com=PrintFriendly.com
PritTorrent=PritTorrent
PrivacyAwareBot=PrivacyAwareBot
PrivacyScore.org crawler=PrivacyScore.org crawler
Prlog=Prlog
probethenet.com scanner=probethenet.com scanner
Project 25499=Project 25499
Promotion_Tools=Promotion_Tools
Protopage=Protopage
proximic=proximic
psbot=psbot
PubMatic Crawler=PubMatic Crawler
Pulsepoint crawler=Pulsepoint crawler
Pulsepoint=Pulsepoint
purebot=purebot
python-requests=python-requests
Python-urllib=Python-urllib
Qwantify=Qwantify
R6 bot=R6 bot
Rambler bot=Rambler bot
RankActiveLinkBot=RankActiveLinkBot
RankingBot=RankingBot
RankurBot=RankurBot
RavenCrawler=RavenCrawler
REDbot=REDbot
redditbot=redditbot
RegionStuttgartBot=RegionStuttgartBot
Re-re crawler=Re-re crawler
RetrevoPageAnalyzer=RetrevoPageAnalyzer
Riddlerbot=Riddlerbot
Rigor crawler=Rigor crawler
Rivva=Rivva
Robots_Tester=Robots_Tester
RobotsChecker=RobotsChecker
rogerbot=rogerbot
Roistat Bot=Roistat Bot
ROR Sitemap Generator=ROR Sitemap Generator
RSiteAuditor=RSiteAuditor
RSSingBot=RSSingBot
RSSMicro.com RSS/Atom Feed Robot=RSSMicro.com RSS/Atom Feed Robot
RukiCrawler=RukiCrawler
Runet-Research-Crawler=Runet-Research-Crawler
rushBot=rushBot
RyowlEngine=RyowlEngine
Ryte Bot=Ryte Bot
s4a=s4a
SabsimBot=SabsimBot
SafeDNSBot=SafeDNSBot
SafeSearch microdata crawler=SafeSearch microdata crawler
SBL-BOT=SBL-BOT
SBSearch=SBSearch
SCFCrawler=SCFCrawler
ScoutJet=ScoutJet
Scrapy=Scrapy
Screaming Frog SEO Spider=Screaming Frog SEO Spider
screeenly-bot=screeenly-bot
scribdbot=scribdbot
seekbot=seekbot
Seeker=Seeker
Seekport Crawler=Seekport Crawler
SeeWithKids bot=SeeWithKids bot
semanticbot=semanticbot
SemanticJuice crawler=SemanticJuice crawler
SemanticScholarBot=SemanticScholarBot
semantic-visions.com crawler=semantic-visions.com crawler
SEMrushBot=SEMrushBot
Semtix.cz bot=Semtix.cz bot
SentiBot=SentiBot
sentry=sentry
Seo Servis=Seo Servis
Seobility=Seobility
SEOdiver=SEOdiver
SEOkicks=SEOkicks
SEOkicks-Robot=SEOkicks-Robot
SeopultContentAnalyzer=SeopultContentAnalyzer
seoscanners.net bot=seoscanners.net bot
seoscanners=seoscanners
SeoSiteCheckup=SeoSiteCheckup
SerendeputyBot=SerendeputyBot
serpstatbot=serpstatbot
SERPtimizerBot=SERPtimizerBot
Server Density=Server Density
ServiceUptime.robot=ServiceUptime.robot
SetCronJob=SetCronJob
Seznam Zbozi.cz=Seznam Zbozi.cz
seznambot=seznambot
Sideqik crawler=Sideqik crawler
SimpleCrawler=SimpleCrawler
SimpleScraper=SimpleScraper
SimplyFast.info=SimplyFast.info
sistrix crawler=sistrix crawler
SISTRIX Optimizer=SISTRIX Optimizer
Site24x7=Site24x7
sitebot=sitebot
SiteCheckerBot=SiteCheckerBot
Sitedomain-Bot=Sitedomain-Bot
siteexplorer.info=siteexplorer.info
SiteExplorer=SiteExplorer
Siteimprove.com bot=Siteimprove.com bot
Siteimprove.com=Siteimprove.com
SiteProbe=SiteProbe
Site-Shot=Site-Shot
SiteTruth.com=SiteTruth.com
SiteUptime.com=SiteUptime.com
SJUUPBot=SJUUPBot
SKB Kontur bot=SKB Kontur bot
SkypeUriPreview=SkypeUriPreview
Slackbot=Slackbot
Slack-ImgProxy=Slack-ImgProxy
Slurp=Slurp
smtbot=smtbot
Snacktory=Snacktory
snapchat-proxy=snapchat-proxy
SnapSearch=SnapSearch
sniptracker=sniptracker
SocialRankIOBot=SocialRankIOBot
sogou spider=sogou spider
Sogou=Sogou
SOLOFIELD bot=SOLOFIELD bot
Sonic=Sonic
SpazioDati bot=SpazioDati bot
spbot=spbot
Specificfeeds=Specificfeeds
speedy=speedy
SpiderLing=SpiderLing
Splash=Splash
SpuhexBot=SpuhexBot
SpyOnWeb=SpyOnWeb
SSL Labs=SSL Labs
ssl-tools.net=ssl-tools.net
sstbot=sstbot
startmebot=startmebot
statdom.ru/Bot=statdom.ru/Bot
StatOnlineRuBot=StatOnlineRuBot
StatusCake=StatusCake
Statusdroid=Statusdroid
Steeler=Steeler
Stephan Kopp bot=Stephan Kopp bot
StorygizeBot=StorygizeBot
Streamline3Bot=Streamline3Bot
Structured Data Markup Helper=Structured Data Markup Helper
Sucuri bot=Sucuri bot
summify=summify
Super Monitoring=Super Monitoring
SuperPagesUrlVerifyBot=SuperPagesUrlVerifyBot
SurdotlyBot=SurdotlyBot
SurveyBot=SurveyBot
SvetaBot=SvetaBot
SWIMGBot=SWIMGBot
Synthesio Crawler=Synthesio Crawler
Sysomos bot=Sysomos bot
Sysomos=Sysomos
TA SEO Crawler=TA SEO Crawler
tagoobot=tagoobot
TangibleeBot=TangibleeBot
Taringa=Taringa
TelegramBot=TelegramBot
Teoma=Teoma
Testomatobot=Testomatobot
TestURI crawler=TestURI crawler
TextRazor crawler=TextRazor crawler
The Knowledge AI=The Knowledge AI
theoldreader.com=theoldreader.com
Thinklab=Thinklab
Thither.Direct Bot=Thither.Direct Bot
thumbshots-de-Bot=thumbshots-de-Bot
TigerBot=TigerBot
TinEye=TinEye
Tiny Tiny RSS=Tiny Tiny RSS
TMM Crawler=TMM Crawler
TomBot=TomBot
Tools4noobs.com Spider=Tools4noobs.com Spider
toplistbot=toplistbot
ToutiaoSpider=ToutiaoSpider
Toweyabot=Toweyabot
Traackr.com=Traackr.com
tracemyfile=tracemyfile
trendictionbot=trendictionbot
Trove=Trove
TurnitinBot=TurnitinBot
TweetmemeBot=TweetmemeBot
Tweezler=Tweezler
twengabot=twengabot
Twingly Recon=Twingly Recon
Twingly=Twingly
Twitterbot=Twitterbot
Twurly=Twurly
UASlinkChecker=UASlinkChecker
uclassify.com=uclassify.com
uMBot=uMBot
umich.edu crawler=umich.edu crawler
um-LN=um-LN
Unshorten.It=Unshorten.It
updown.io bot=updown.io bot
Upflow=Upflow
uptime.com=uptime.com
Uptimebot.org=Uptimebot.org
Uptimebot=Uptimebot
UptimeRobot=UptimeRobot
Uptimia=Uptimia
uptm.io=uptm.io
urlappendbot=urlappendbot
Urlcheckr=Urlcheckr
URLitor.com=URLitor.com
useeBookChecker=useeBookChecker
UsineNouvelleCrawler=UsineNouvelleCrawler
Vagabondo=Vagabondo
Validator.nu=Validator.nu
vebidoobot=vebidoobot
VelenPublicWebCrawler=VelenPublicWebCrawler
Veoozbot=Veoozbot
VerticalLeap bot=VerticalLeap bot
veu=veu
Virusdie crawler=Virusdie crawler
virustotal=virustotal
vkShare=vkShare
voilabot=voilabot
Vsevjednom.cz crawler=Vsevjednom.cz crawler
vuhuvBot=vuhuvBot
W3C_CSS_Validator=W3C_CSS_Validator
W3C_I18n-Checker=W3C_I18n-Checker
W3C_Unicorn=W3C_Unicorn
W3C_Validator=W3C_Validator
W3C-checklink=W3C-checklink
W3C-mobileOK=W3C-mobileOK
w3dt.net crawler=w3dt.net crawler
Wappalyzer=Wappalyzer
WatchMouse=WatchMouse
wbsearchbot=wbsearchbot
Web Manifest Validator=Web Manifest Validator
webarchiv.cz=webarchiv.cz
web-archive-net.com.bot=web-archive-net.com.bot
WebBot=WebBot
webcompanycrawler=webcompanycrawler
WebCookies=WebCookies
WebCrawler=WebCrawler
Webcron.org crawler=Webcron.org crawler
WebDataStats=WebDataStats
webdepozit.sk crawler=webdepozit.sk crawler
WebGazer=WebGazer
webmon=webmon
weborama crawler=weborama crawler
Webshot=Webshot
websitepulse bot=websitepulse bot
WebSniffer=WebSniffer
WebThumb=WebThumb
WEDOS OnLine monitoring=WEDOS OnLine monitoring
WeSEE=WeSEE
WGETbot=WGETbot
WhatCMS.org bot=WhatCMS.org bot
WhatsApp=WhatsApp
WhoAPI=WhoAPI
WikiDo=WikiDo
Wikipedia crawler=Wikipedia crawler
wmtips.com=wmtips.com
wocbot=wocbot
woobot=woobot
WordPress.com bots=WordPress.com bots
WordupInfoSearch=WordupInfoSearch
woriobot=woriobot
WormlyBot=WormlyBot
wotbox=wotbox
WPSec=WPSec
WURFL ImageEngine=WURFL ImageEngine
WWRanks.com crawler=WWRanks.com crawler
Xenu Link Sleuth=Xenu Link Sleuth
XML Schema Validator=XML Schema Validator
XML Sitemaps Generator=XML Sitemaps Generator
XmlSitemapGenerator=XmlSitemapGenerator
xovibot=xovibot
yacybot=yacybot
Yahoo=Yahoo
Yandex.Metrica=Yandex.Metrica
Yandex.Webmaster=Yandex.Webmaster
YandexAccessibilityBot=YandexAccessibilityBot
YandexBot=YandexBot
YandexImages=YandexImages
YandexMobileBot=YandexMobileBot
yanga=yanga
YellowLabTools=YellowLabTools
Yeti=Yeti
YFF35=YFF35
YioopBot=YioopBot
YisouSpider=YisouSpider
YoozBot=yoozBot
YP crawler=YP crawler
Zabbix=Zabbix
zgrab=zgrab
Zombiebot=Zombiebot
ZoomBot=ZoomBot
Zoominfo=Zoominfo
Zoxh.Com Bot=Zoxh.Com Bot
ZumBot=ZumBot
ZuperlistBot=ZuperlistBot

Fresh Installs of UBB.threads 7.6.2 to 7.7.3 are pre-populated with this list.
The 2014-10-14 List:
Code
008=008
192.com=192.com
200PleaseBot=200PleaseBot
360Spider=360Spider
4seohuntBot=4seohuntBot
50.nu=50.nu
^Nail=^Nail
A6-Indexer=A6-Indexer
abby=abby
Aboundexbot=Aboundexbot
AboutUsBot=AboutUsBot
Abrave Spider=Abrave Spider
Accelobot=Accelobot
AcoonBot=Acoon
AddThis.com=AddThis.com robot
ADmantX=ADmantX
adressendeutschland.de=adressendeutschland.de
AdsBot-Google=AdsBot-Google
AhrefsBot=AhrefsBot
aiHitBot=aiHitBot
akula=akula
alexa site audit=alexa site audit
Alexabot=Alexabot
Amagit.COM=Amagit.COM
amibot=amibot
AMZNKAssocBot=AMZNKAssocBot
AntBot=AntBot
Apercite=Apercite
AportWorm=AportWorm
AraBot=AraBot
arachnode.net=arachnode.net
Arachnophilia=Arachnophilia
archive.org arc=special_archiver
archive.org bot=archive.org_bot
Ask Jeeves=Ask Jeeves
AskQuickly=AskQuickly
Automattic Analytics Crawler=Automattic Analytics Crawler
BabalooSpider=BabalooSpider
backlink-check.de=backlink-check.de
BacklinkCrawler=BacklinkCrawler
Bad-Neighborhood=Bad Neighborhood Header Detector
Bad-Neighborhood=Bad-Neighborhood Link Analyzer
Baiduspider JP=Baiduspider japan
Baiduspider-Image=Baiduspider-image
Baiduspider=Baiduspider
baypup=baypup
BDCbot=BDCbot
BDFetch=BDFetch
BegunAdvertising=BegunAdvertising
bingbot=bingbot
bingbot=bingbot SitemapProbe
BingPreview=BingPreview
bitlybot=bitlybot
biwec=biwec
bixocrawler=adbeat-publisher-description-fetcher
bixocrawler=bixo
bixocrawler=bixocrawler
bixocrawler=bixolabs
bixocrawler=ptd-crawler
bl.uk_lddc_bot=bl.uk_lddc_bot
Blekkobot=Blekkobot
BLEXBot=BLEXBot
BlinkaCrawler=BlinkaCrawler
BlogPulse=BlogPulse
BlogPulse=BlogPulseLive
bnf.fr_bot=bnf.fr_bot
bot-pge.chlooe.com=bot-pge.chlooe.com
bot.wsowner.com=bot.wsowner.com
botmobi=botmobi
BotOnParade=BotOnParade
BrainbruBot=BrainbruBot
Browsershots=Browsershots
BUbiNG=BUbiNG
Butterfly=Butterfly
CamontSpider=CamontSpider
CareerBot=CareerBot
Castabot=Castabot
CatchBot=CatchBot
CCBot=CCBot
CCResearchBot=CCResearchBot
ChangeDetection=ChangeDetection
Charlotte=Charlotte
CirrusExplorer=CirrusExplorer
cityreview=cityreview
classbot=classbot
CligooRobot=CligooRobot
CliqzBot=Cliqz
CloudFlare-AlwaysOnline=CloudFlare-AlwaysOnline
CloudServerMarketSpider=CloudServerMarketSpider
CMS Crawler=CMS Crawler
coccoc=coccoc
COMODOSpider=Comodo
CompSpyBot=CompSpyBot
contentDetection=contentDetection
copyright sheriff=copyright sheriff
CorpusCrawler=CorpusCrawler
Covario-IDS=Covario
crawler for netopian=crawler for netopian
Crawler4j=Crawler4j
CrazyWebCrawler=crazywebcrawler
Crowsnest=Crowsnest
Curious George=Curious George
datagnionbot=datagnionbot
Daumoa=Daumoa
DBLBot=DBLBot
DCPbot=DCPbot
DealGates Bot=DealGates Bot
discoverybot=discobot
discoverybot=discoverybot
DKIMRepBot=DKIMRepBot
dlcbot=dlcbot
dlvr.it=dlvr.it
DNS-Digger-Explorer=DNS-Digger-Explorer
DomainAppender=DomainAppender
DomainDB=DomainDB
DomainTools-HeaderCheck=HTTP Headers Online
DomainTools-LinksCheck=Online Website Link Checker
DomainTools-SitemapGen=Online Sitemap Generator
DotBot=DotBot
dotSemantic=dotSemantic
DripfeedBot=DripfeedBot
drupact=drupact
DuckDuckPreview=DuckDuckPreview
EasouSpider=EasouSpider
EasyBib AutoCite=EasyBib AutoCite
eCairn-Grabber=eCairn-Grabber
eCommerceBot=eCommerceBot
EdisterBot=EdisterBot
Embedly=Embedly
emefgebot=emefgebot
envolk=envolk
Esribot=Esribot
EuripBot=EuripBot
Eurobot=Eurobot
EventGuruBot=EventGuruBot
EveryoneSocialBot=EveryoneSocialBot
EvriNid=EvriNid
Exabot Images=Exabot-Images
Exabot Thumbnails=Exabot-Thumbnails
Exabot=Exabot
Exabot=ExaleadCloudview
Experibot=Experibot
Ezooms=Ezooms
FacebookExternalHit=FacebookExternalHit
facebookPlatform=facebookplatform
factbot=Factbot 1.09
FairShare=FairShare
Falconsbot=Falconsbot
fastbot crawler=fastbot crawler
FauBot=FauBot
FeedCatBot=FeedCatBot
FeedFinder=FeedFinder
Feedly=Feedly
Feedly=FeedlyBot
Fetch-Guess=Fetch
findlinks=findlinks
firmilybot=firmilybot
Flatland Industries Web Spider=flatlandbot
FlightDeckReportsBot=FlightDeckReportsBot
FlipboardProxy=FlipboardProxy
Flocke bot=Flocke bot
FollowSite Bot=FollowSite Bot
Fooooo_Web_Video_Crawl=Fooooo_Web_Video_Crawl
FreeWebMonitoring=FreeWebMonitoring
FyberSpider=FyberSpider
Gaisbot=Gaisbot
GarlikCrawler=GarlikCrawler
GeliyooBot=GeliyooBot
Genieo Web filter=Genieo
GigablastOpenSource=GigablastOpenSource
Gigabot=Gigabot
GingerCrawler=GingerCrawler
Girafabot=Girafabot
gocrawl=gocrawl
GOFORITBOT=GOFORITBOT
gonzo=gonzo
Googlebot AdsBot Mobile=AdsBot-Google-Mobile
Googlebot AdSense=Mediapartners-Google
Googlebot Image=Googlebot-Image
Googlebot Mobile=Googlebot-Mobile
Googlebot Snippet=Googlebot snippet
Googlebot Video=Googlebot-Video
Googlebot Web Preview=Google Web Preview
Googlebot=Googlebot
Grahambot=Grahambot
GrapeshotCrawler=grapeFX
GrapeshotCrawler=GrapeshotCrawler
GurujiBot=GurujiBot
Hailoobot=Hailoobot
HatenaScreenshot=HatenaScreenshot
hawkReader=hawkReader
HeartRails_Capture=HeartRails_Capture
heritrix=heritrix
Holmes=holmes
HolmesBot=HolmesBot
HomeTags=HomeTags
HostTracker.com=HostTracker.com
HostTracker=HostTracker
HuaweiSymantecSpider=HuaweiSymantecSpider
HubSpot Connect=HubSpot Connect
HubSpot Crawler=HubSpot Crawler
HypeStat=HypeStat
ia_archiver=ia_archiver
ICC-Crawler=ICC-Crawler
ichiro=ichiro
iCjobs=iCjobs
IdeelaborPlagiaat=IdeelaborPlagiaat
idmarch=idmarch Automatic.beta
Iframely=Iframely
imbot=imbot
immediatenet thumbnails=immediatenet thumbnails
ImplisenseBot=ImplisenseBot
Impressumscrawler=Impressumscrawler
Influencebot=Influencebot
Infohelfer=Infohelfer
IntegromeDB=IntegromeDB
IstellaBot=IstellaBot
IXEbot=IXEbot
Jabse.com Crawler=Jabse.com Crawler
JadynAveBot=JadynAveBot
JamesBOT=JamesBOT
JikeSpider=JikeSpider
Job Roboter Spider=Job Roboter Spider
JUST-CRAWLER=JUST-CRAWLER
Jyxobot=Jyxobot
Jyxobot=JyxobotRSS
Kalooga=Kalooga
Karneval-Bot=Karneval-Bot
kinshoo=KiNShooboT
KomodiaBot=KomodiaBot
Kraken=Kraken
KrOWLer=KrOWLer
kulturarw=kulturarw3
L.webis=L.webis
Leikibot=Leikibot
LemurWebCrawler=LemurWebCrawler
LexxeBot=LexxeBot
Lijit=Lijit
LinguaBot=LinguaBot
linguatools=linguatools
Linguee Bot=Linguee Bot
Link Valet Online=Link Valet Online
LinkAider=LinkAider
linkdex.com=linkdex.com
linkdexbot=linkdexbot
LinkedInBot=LinkedInBot
LinkWalker=LinkWalker
Lipperhey Spider=Lipperhey Spider
livedoor ScreenShot=livedoor ScreenShot
LivelapBot=LivelapBot
LoadImpactPageAnalyzer=LoadImpactPageAnalyzer
LoadTimeBot=LoadTimeBot
LuminateBot=LuminateBot
magpie-crawler=magpie-crawler
Mail.Ru bot=Mail.Ru
meanpathbot=meanpathbot
MeMoNewsBot=MeMoNewsBot
memoryBot=memoryBot
MetaGeneratorCrawler=MetaGeneratorCrawler
MetaHeadersBot=MetaHeadersBot
MetaJobBot=MetaJobBot
MetamojiCrawler=MetamojiCrawler
Metaspinner=Metaspinner
MetaURI API=MetaURI API
MetaURI=MetaURI
MIA Bot=MIA Bot
MiaDev=MiaDev
MixBot=MixBot
MJ12bot=MJ12bot
MLBot=MLBot
MnoGoSearch=MnoGoSearch
Moatbot=Moatbot
moba-crawler=moba-crawler
MojeekBot=MojeekBot
Motoricerca-Robots.txt-Checker=Motoricerca-Robots.txt-Checker
Mp3Bot=Mp3Bot
MSNBot Media=msnbot-media
MSNBot News=msnbot-NewsBlogs
MSNBot UDiscovery=msnbot-UDiscovery
MSNBot=adidxbot
MSNBot=MSNBot
MSRBOT=MSRBOT
musobot=musobot
Najdi.si=Najdi.si
NalezenCzBot=NalezenCzBot
NaverBot=NaverBot
NaverBot=Yepi
NaverBot=Yeti
NaverBot=Yeti-FeedItemCrawler
NaverBot=Yeti-Mobile
nekstbot=nekstbot
NerdByNature.Bot=NerdByNature.Bot
NerdyBot=NerdyBot
NetcraftSurveyAgent=NetcraftSurveyAgent
netEstate Crawler=netEstate NE Crawler
netEstate Crawler=netEstate RSS crawler
NetResearchServer=nrsbot
Netseer=Netseer crawler
NextGenSearchBot=NextGenSearchBot
Nigma.ru=Nigma.ru
NLNZ_IAHarvester2013=NLNZ_IAHarvester2013
nodestackbot=nodestackbot
Nuhk=Nuhk
Nutch=Nutch
nworm=nwormFeedFinder
Nymesis=Nymesis
oBot=oBot
Ocelli=Ocelli
omgilibot=omgilibot
OoyyoBot=OoyyoBot
Open Web Analytics Bot=Open Web Analytics Bot
OpenCalaisSemanticProxy=OpenCalaisSemanticProxy
OpenindexSpider=OpenindexDeepSpider
OpenindexSpider=OpenindexShalooowSpider
OpenindexSpider=OpenindexSpider
OpenWebSpider=OpenWebSpider
OrgbyBot=OrgbyBot
OsObot=OsObot
ownCloud Server Crawler=ownCloud Server Crawler
Page2RSS=Page2RSS
page_verifier=page_verifier
PagePeeker=PagePeeker
Panscient web crawler=Panscient web crawler
PaperLiBot=PaperLiBot
ParchBot=ParchBot
parsijoo=parsijoo
PayPal IPN=PayPal IPN
Peeplo Screenshot Bot=Peeplo Screenshot Bot
Peepowbot=Peepowbot
peerindex=peerindex
Peew=Peew
PercolateCrawler=percbotspider
PercolateCrawler=PercolateCrawler
pingdom.com_bot=pingdom.com_bot
Pinterest=Pinterest
PiplBot=PiplBot
Pixray-Seeker=Pixray-Seeker
Plukkie=Plukkie
pmoz.info ODP link checker=pmoz.info ODP link checker
Pompos=Pompos
PostPost=PostPost
pr-cy.ru Screenshot Bot=pr-cy.ru
ProCogBot=ProCogBot
ProCogSEOBot=ProCogSEOBot
proximic=proximic
psbot=psbot
psbot=psbot-page
Qirina Hurdler=Qirina Hurdler
Qseero=Qseero
Qualidator.com Bot=Qualidator.com Bot
Qualidator.com SiteAnalyzer=Qualidator.com SiteAnalyzer
QuerySeekerSpider=QuerySeekerSpider
quickobot=quickobot
R6 bot=R6_CommentReader
R6 bot=R6_FeedFetcher
RADaR-Bot=RADaR-Bot
RankurBot=RankurBot
Readability=Readability
Robots_Tester=Robots_Tester
Robozilla=Robozilla
rogerbot=rogerbot
Ronzoobot=Ronzoobot
RSSMicro.com RSS Atom Feed Robot=RSSMicro.com RSS Atom Feed Robot
Ruky-Roboter=Ruky-Roboter
RyzeCrawler=RyzeCrawler
SAI Crawler=SAI Crawler
SanszBot=SanszBot
SBIder=SBIder
SBSearch=SBSearch
Scarlett=Scarlett
SCFCrawler=SCFCrawler
Scooter=Scooter
ScoutJet=ScoutJet
ScoutJet=ScoutJet old
Scrapy=Scrapy
ScreenerBot Crawler=ScreenerBot Crawler
Scrubby=Scrubby
search.KumKie.com=search.KumKie.com
Search17Bot=Search17Bot
SearchmetricsBot=SearchmetricsBot
SecurityResearchBot=SecurityResearchBot
seegnifybot=seebot
seegnifybot=seegnifybot
Semager=Semager
Semantifire=Semantifire1
SemrushBot=SemrushBot
Seobility=Seobility SEO-Check
Seobility=Seobility Urlstat
SEOCentro Keywords=KeywordDensityRobot
SEOCentro MetaTags=MetaTagRobot
SEOCentro SEO Keywords=SEOCentro Page Keyword Analyzer v1.2
SeoCheckBot=SeoCheck
SEODat=SEODat
SEOdiver=SEOdiver
SEOENGBot=SEOENGBot
SEOkicks-Robot=SEOkicks-Robot
Setoozbot=OOZBOT
Setoozbot=SETOOZBOT
SeznamBot=Seznam screenshot-generator
SeznamBot=SeznamBot
SeznamBot=SklikBot
Shareaholicbot=Shareaholicbot
Shelob=Shelob
ShopWiki=ShopWiki
ShowyouBot=ShowyouBot
sistrix=sistrix
SiteCondor=SiteCondor
Sitedomain-Bot=Sitedomain-Bot
Slackbot=Slackbot-LinkExpanding
smart.apnoti.com Robot=smart.apnoti.com Robot
SMTBot=SMTBot
Snapbot=Snapbot
SniffRSS=SniffRSS
socialbm_bot=socialbm_bot
sogou spider=Sogou
SolomonoBot=SolomonoBot
Sosospider=Sosoimagespider
Sosospider=Sosospider
spbot=spbot
Speedy=Speedy Spider
Speedy=Speedy Spider Beta
SpiderLing=SpiderLing
Spinn3r=Spinn3r
SputnikBot=SputnikBot
SSL-Crawler=SSL-Crawler
SSLBot=SSLBot
StackRambler=StackRambler
StatoolsBot=StatoolsBot
Steeler=Steeler
STINGbot=STINGbot
stq_bot=stq_bot
Strokebot=Strokebot
suggybot=suggybot
SurcentroBot=SurcentroBot
Surphace Scout=Surphace Scout
SurveyBot=SurveyBot
SWEBot=SWEBot
SygolBot=SygolBot
Symfony Spider=Symfony Spider
Szukacz=Szukacz
Tagoobot=Tagoobot
taptubot=taptubot
Technoratibot=Technoratibot
Thumbnail.CZ robot=Thumbnail.CZ robot
ThumbShots-Bot=ThumbShots-Bot
thumbshots-de-Bot=thumbshots-de-bot
Thumbshots.ru=Thumbshots.ru
ThumbSniper=ThumbSniper
TinEye=TinEye
TomTom places company search=TomTom places company search
Topicbot=Topicbot
Toread-Crawler=Toread-Crawler
Touche=Touche
trendictionbot=trendictionbot
TurnitinBot=TurnitinBot
TwengaBot=TwengaBot
TwengaBot=TwengaBot-Discover
Twiceler=Twiceler
Twikle=Twikle
Twingly Recon=Twingly Recon
UASlinkChecker=UASlinkChecker
uMBot FC=uMBot-FC
uMBot LN=uMBot-LN
UnisterBot=UnisterBot
UnwindFetchor=UnwindFetchor
Updownerbot=Updownerbot
UptimeDog=UptimeDog
UptimeRobot=UptimeRobot
URLAppendBot=URLAppendBot
urlfan-bot=urlfan-bot
Urlfilebot (Urlbot)=Urlfilebot
Vagabondo=Vagabondo
Vedma=Vedma
VideoSurf_bot=VideoSurf_bot
Visbot=Visbot
VoilaBot=VoilaBot
voltron=voltron
voyager=voyager
WASALive-Bot= WASALive-Bot
WatchMouse=WatchMouse
WBSearchBot=WBSearchBot
Web-Monitoring=Web-Monitoring
Web-sniffer=Web-sniffer
WebCookies=WebCookies
WebCorp=WebCorp
WebImages=WebImages
webinatorbot=webinatorbot
webmastercoffee=webmastercoffee
WebNL=WebNL
WebRankSpider=WebRankSpider
WebTarantula.com Crawler=WebTarantula.com Crawler
WebThumbnail=WebThumbnail
WebWatch Robot_txtChecker=WebWatch
WeSEE:Ads=WeSEE:Ads/PageBot
WeSEE:Ads=WeSEE:Ads/PictureBot
WeSEE:Search=WeSEE
WeSEE:Search=WeSEE:Search
WeViKaBot=WeViKaBot
Whoismindbot=Whoismindbot
WikioFeedBot=WikioFeedBot
wikiwix-bot=wikiwix-bot
Willow Internet Crawler=Willow Internet Crawler
WillyBot=WillyBot
WinWebBot=WinWebBot
WMCAI_robot=WMCAI_robot
Woko=Woko
WordPress.com mShots=WordPress.com mShots
woriobot=woriobot
Wotbox=Wotbox
wsAnalyzer=wsAnalyzer
wscheck.com=wscheck.com
x28-job-bot=x28-job-bot
XmarksFetch=XmarksFetch
XML Sitemaps Generator=XML Sitemaps Generator
XoviBot=XoviBot
XRL=XRL
Yaanb=Yaanb
yacybot=yacybot
Yahoo!=Y!J-BRI
Yahoo!=Y!J-BRJ
Yahoo!=Y!J-BRO
Yahoo!=Y!J-BRW
Yahoo!=Y!J-BSC
Yahoo!=Yahoo! Slurp
Yahoo!=Yahoo-MMCrawler
Yahoo!=YahooCacheSystem
YamanaLab-bot=Sonic
YandexBot AntiVirus=YandexAntivirus
YandexBot Blogs=YandexBlogs
YandexBot Catalog=YandexCatalog
YandexBot Direct=YandexDirect
YandexBot Favicons=YandexFavicons
YandexBot ImageResizer=YandexImageResizer
YandexBot Images=YandexImages
YandexBot Media=YandexMedia
YandexBot Metrix=YandexMetrika
YandexBot News=YandexNews
YandexBot Server=Yandex.Server
YandexBot Something=YandexSomething
YandexBot Video=YandexVideo
YandexBot Webmaster=YandexWebmaster
YandexBot Zakladki=YandexZakladki
YandexBot=Yandex
Yanga=Yanga
YioopBot=gofind
YioopBot=YioopBot
YodaoBot Image=YodaoBot-Image
YodaoBot=YodaoBot
YoudaoBot=YoudaoBot
YowedoBot=YowedoBot
YRSpider=YRSpider
YYSpider=YYSpider
ZeerchBot LA1=ZeerchBot LA1
ZeerchBot LA2=ZeerchBot LA2
ZeerchBot=ZeerchBot
Zookabot=Zookabot
ZumBot=ZumBot

Fresh Installs prior to UBB.threads 7.6.2 do not come prepopulated with a Search Engine Agents list.
Liked Replies
by isaac
isaac
Although the source information for you to create your own updates and conversions is in the OP, I plan on updating this post every couple of months, making your job as an UBBT forum admin much easier.

---
EDIT: user-agent-string.info no longer provides a list of user agent strings without a subscription. this means that until another resource for this data is found, this list will remain as it currently is.
1 member likes this
by isaac
isaac
This list has been updated 2014-10-14.
1 member likes this
by isaac
isaac
Some further reading regarding http/https referer data:
https://yoast.com/web-https/
1 member likes this
by isaac
isaac
SteveS, are you confident your Amazon stuff is not related caching of content within your site by AWS Cloud Computing /Route 53? https://aws.amazon.com/

Additional reading at:
https://en.wikipedia.org/wiki/Amazon_Route_53

The IP 72.21.217.n has been used as a proxy for a User Agent of MSIE-6, which is in itself highly deprecated. Headers can also be consistent with either a battened-down proxy or a bot.

Additional reading at:
"amazonaws.com plays host to wide variety of bad bots"
https://www.webmasterworld.com/search_engine_spiders/3828718.htm
1 member likes this
by isaac
isaac
Changelog 2020-01-14
The Robots (Crawlers) List for UBB.threads has been updated in OP
1 member likes this
by isaac
isaac
Changelog 2020-09-24
The Robots (Crawlers) List for UBB.threads has been updated in OP
1 member likes this
Donate Today!
Donate via PayPal

Donate to UBBDev today to help aid in Operational, Server and Script Maintenance, and Development costs.

Please also see our parent organization VNC Web Services if you're in the need of a new UBB.threads Install or Upgrade, Site/Server Migrations, or Security and Coding Services.
Recommended Hosts
We have personally worked with and recommend the following Web Hosts:
Stable Host
bluehost
InterServer
Visit us on Facebook
Member Spotlight
badfrog
badfrog
somewhere on the coast of Maine
Posts: 94
Joined: March 2007
Forum Statistics
Forums63
Topics37,551
Posts293,823
Members13,839
Most Online5,166
Sep 15th, 2019
Today's Statistics
Currently Online 307
Topics Created 0
Posts Made 0
Users Online 0
Birthdays 26
Top Posters
AllenAyres 21,079
JoshPet 10,369
LK 7,394
Lord Dexter 6,708
Gizmo 5,808
Greg Hard 4,625
Top Posters(30 Days)
isaac 5
Top Likes Received
isaac 64
Gizmo 12
Brett 7
WebGuy 2
Top Likes Received (30 Days)
isaac 3
The UBB.Developers Network (UBB.Dev/Threads.Dev) is ©2000-2020 VNC Web Services

 
Powered by UBB.threads™ PHP Forum Software 7.7.6
(Snapshot build 20201121.dev)
Responsive Width:

PHP: 5.4.45 Page Time: 8.761s Queries: 8 (8.748s) Memory: 3.0128 MB (Peak: 3.0654 MB) Data Comp: Zlib Server Time: 2020-11-26 23:23:45 UTC
Valid HTML 5 and Valid CSS