AI Bots Lists
The 3 first lists are included in SecuPress, the last one "uncategorized" is not.
TOP 20 Most Common AI Bot User-Agent
- Amazonbot
- anthropic-ai
- Applebot
- Applebot-Extended
- Bytespider
- CCBot
- ChatGPT-User
- Claude-Web
- ClaudeBot
- Diffbot
- FacebookBot
- Google-Extended
- GPTBot
- Kangaroo Bot
- Meta-ExternalAgent
- Meta-ExternalFetcher
- OAI-SearchBot
- PerplexityBot
- Timpibot
- YouBot
TOP 40 Most Common AI Bot User-Agents
- AI2Bot
- Ai2Bot-Dolma
- Amazonbot
- anthropic-ai
- Applebot
- Applebot-Extended
- Bytespider
- CCBot
- ChatGPT-User
- Claude-Web
- ClaudeBot
- cohere-ai
- Diffbot
- FacebookBot
- facebookexternalhit
- FriendlyCrawler
- Google-Extended
- GoogleOther
- GoogleOther-Image
- GoogleOther-Video
- GPTBot
- iaskspider/2.0
- ICC-Crawler
- ImagesiftBot
- img2dataset
- ISSCyberRiskCrawler
- Kangaroo Bot
- Meta-ExternalAgent
- Meta-ExternalFetcher
- OAI-SearchBot
- omgili
- omgilibot
- PerplexityBot
- PetalBot
- Scrapy
- Sidetrade indexer bot
- Timpibot
- VelenPublicWebCrawler
- Webzio-Extended
- YouBot
TOP Most Known AI Bot User-Agents (~200)
- 008
- 2ip bot
- 360Spider
- adbeat_bot
- AdsBot-Google
- AdsBot-Google-Mobile
- AhrefsBot
- AhrefsSiteAudit
- AI2Bot
- aiHitBot
- AlexandriaOrgBot
- Amazonbot
- AndersPinkBot
- anthropic-ai
- Applebot
- Applebot-Extended
- archive.org_bot
- ArchiveBot
- Arquivo-web-crawler
- Atom Feed Robot
- AwarioBot
- AwarioSmartBot
- Baiduspider
- Barkrowler
- BazQux
- bingbot
- BingPreview
- bitlybot
- BitSightBot
- Blackboard
- BLEXBot
- BrandVerity
- BrightEdge Crawler
- BublupBot
- Bytespider
- CCBot
- ChatGPT-User
- Chrome-Lighthouse
- Cincraw
- Claude-Web
- ClaudeBot
- coccocbot-web
- Cocolyzebot
- cohere-ai
- Dark Visitor
- DataForSeoBot
- Dataprovider.com
- Daum
- dcrawl
- deadlinkchecker
- Diffbot
- Discordbot
- DomainStatsBot
- dotbot
- DuckDuckBot
- DuckDuckGo-Favicons-Bot
- Embedly
- ev-crawler
- FacebookBot
- facebookexternalhit
- Feedfetcher-Google
- Feedly
- FlipboardProxy
- FreshRSS
- Friendica
- Google Favicon
- Google Web Preview
- Google-Extended
- Google-InspectionTool
- Google-Read-Aloud
- Google-Safety
- Googlebot
- Googlebot-Image
- Googlebot-Mobile
- Googlebot-News
- Googlebot-Video
- GoogleOther
- GPTBot
- HaoSouSpider
- Hatena
- heritrix
- HTTrack
- HTTrack 3.0
- HubSpot
- hypestat
- ia_archiver
- ia_archiver-web.archive.org
- Iframely
- ImagesiftBot
- inoreader
- IonCrawl
- Jugendschutzprogramm-Crawler
- Kangaroo Bot
- KStandBot
- LightspeedSystemsCrawler
- linkdexbot
- LinkedInBot
- linkfluence
- LinkWalker
- magpie-crawler
- Mail.RU_Bot
- Mastodon
- Mediapartners-Google
- Mediatoolkitbot
- Meta-ExternalAgent
- Meta-ExternalFetcher
- MetaInspector
- Miniflux
- MJ12bot
- MojeekBot
- msnbot
- msnbot-media
- MuckRack
- NetcraftSurveyAgent
- Netvibes
- NewsBlur
- newspaper
- Nextcloud
- Nicecrawler
- Nutch
- OAI-SearchBot
- Offline Explorer
- omgili
- online-webceo-bot
- OpenindexSpider
- Pandalytics
- panscient.com
- PerplexityBot
- PetalBot
- Pinterestbot
- PocketParser
- proximic
- Qwantify
- redditbot
- rogerbot
- scoop.it
- Scrapy
- Screaming Frog SEO Spider
- SeekportBot
- SemanticScholarBot
- SemrushBot
- SemrushBot-BA
- SemrushBot-CT
- SemrushBot-SI
- SemrushBot-SWA
- SenutoBot
- SeobilityBot
- SEOkicks
- SEOlizer
- SerendeputyBot
- serpstatbot
- SeznamBot
- SimplePie
- SiteAuditBot
- SiteCheckerBotCrawler
- SkypeUriPreview
- Slackbot-LinkExpanding
- SMTBot
- Snap URL Preview Service
- snapchat
- Sogou web spider
- startmebot
- Superfeedr
- SurdotlyBot
- Synapse
- t3versionsBot
- TelegramBot
- teoma
- Timpibot
- TinEye
- TinEye-bot
- Tiny Tiny RSS
- trendictionbot
- TrendsmapResolver
- Turnitin
- TurnitinBot
- TweetmemeBot
- Twingly
- Twitterbot
- um-LN
- VelenPublicWebCrawler
- Viber
- virustotal
- vkShare
- W3C_CSS_Validator
- W3C_Validator
- Webzio
- Webzio-Extended
- WellKnownBot
- yacybot
- Yahoo Link Preview
- Yahoo! Slurp
- Yandex
- YandexBot
- YandexImages
- YandexRenderResourcesBot
- Yeti
- YisouSpider
- YouBot
- ZoomBot
- ZoominfoBot
- ZumBot
TOP Most Uncategorized AI Bot User-Agents (~600)
- 007ac9 Crawler
- 2ip.ru
- 360Spider-Image
- 360Spider-Video
- 5emeRue
- 5erue
- A Patent Crawler
- A6-Indexer
- Aboundex
- AcademicBotRTU
- acapbot
- acoonbot
- Acunetix Security Scanner
- Acunetix Web Vulnerability Scanner
- AddSearchBot
- AddThis
- adequat
- adequat-systems
- AdIdxBot
- ADmantX
- adscanner
- AdsTxtCrawler
- AdvBot
- AISearchBot
- Alexabot
- Alexibot
- AlphaBot
- AmiSoftware
- antibot
- AnyEvent
- Apercite
- AppInsights
- Aqua_Products
- arabot
- Ask n read
- asknread.com
- AspiegelBot
- asterias
- Augure
- auramundi
- AwarioRssBot
- awesomecrawler
- B2B Bot
- b2w
- BackDoorBot
- BacklinkCrawler
- Baidu-YunGuanCe
- Baiduspider-image
- Baiduspider-news
- Baiduspider-video
- BDCbot
- BehloolBot
- betaBot
- Better Uptime Bot
- bidswitchbot
- BIGLOTRON
- binlar
- Birdcrawlerbot
- BitBot
- Black Hole
- Blekkobot
- blogmuraBot
- BlowFish
- BLP_bbot
- bnf.fr_bot
- BomboraBot
- Bookmark search tool
- bot-pge.chlooe.com
- Bot.AraTurka.com
- BotALot
- botify
- BotRightHere
- BoxcarBot
- brainobot
- BrandONbot
- BTWebClient
- BUbiNG
- Buck
- BuiltBotTough
- Bullseye
- BunnySlippers
- buzzbot
- Caliperbot
- CapsuleChecker
- careerbot
- CC Metadata Scaper
- Cegbfeieh
- centurybot
- changedetection
- CheckMarkNetwork
- CheeseBot
- CherryPicker
- CherryPickerElite
- CherryPickerSE
- Cision
- CISPA Webcrawler
- citeseerxbot
- Citoid
- Claritybot
- Clickagy
- Cliqzbot
- CloudFlare-AlwaysOnline
- coccoc
- coccocbot
- coexel
- Companybook-Crawler
- content crawler spider
- ContextAd Bot
- contxbot
- convera
- ConveraCrawler
- Cookiebot
- Copernic
- CopyRightCheck
- Corporama
- cosmos
- crawler4j
- CrawlyProjectCrawler
- Crescent
- Crescent Internet ToolPak HTTP OLE Control v.1.0
- CriteoBot
- CrunchBot
- CrystalSemanticsBot
- Curebot
- Cutbot
- cXensebot
- CyberPatrol
- DareBoost
- Datafeedwatch
- datagnionbot
- Datanyze
- daumoa
- deepcrawl
- deepnoc
- DeuSu
- Digg Deeper
- Digimind
- Digincore bot
- discobot
- Disqus
- DittoSpyder
- DnyzBot
- Domain Re-Animator Bot
- DomainCrawler
- Dow Jones Searchbot
- Download Ninja
- Dragonbot
- drupact
- Dubbotbot
- e.ventures Investment Crawler
- EasyBib AutoCite
- ec2linkfinder
- edisterbot
- electricmonk
- elisabot
- ellisphere
- EmailCollector
- EmailSiphon
- EmailWolf
- epicbot
- eright
- EroCrawler
- EtaoSpider
- europarchive.org
- evc-batch
- EveryoneSocialBot
- Exabot
- Experibot
- ExtLinksBot
- ExtractorPro
- Eyeotabot
- EZID
- Ezooms
- Facebot
- FairAd Client
- FAST Enterprise Crawler
- FAST-WebCrawler
- FediDB
- fedoraplanet
- Feedbin
- feedbot
- FeedBurner
- Feedspot
- FeedValidator
- FemtosearchBot
- Fever
- FindITAnswersbot
- findlink
- findthatfile
- findxbot
- Flaming AttackBot
- Flamingo_SearchEngine
- fluffy
- Foobot
- fr-crawler
- FreeWebMonitoring SiteChecker
- FreshpingBot
- fuelbot
- Fyrebot
- g00g1e.net
- G2 Web Services
- g2reader-bot
- Gaisbot
- GarlikCrawler
- Genieo
- GenomeCrawlerd
- GetRight
- Gigablast
- Gigabot
- GingerCrawler
- Gluten Free Crawler
- gnam gnam spider
- GnowitNewsbot
- Google-Adwords-Instant
- Google-Certificates-Bridge
- Google-PhysicalWeb
- Google-Site-Verification
- Google-Structured-Data-Testing-Tool
- google-xrawler
- Gowikibot
- grapeshot
- GrapeshotCrawler
- Grobbot
- GroupHigh
- grub-client
- grub.org
- gsa-crawler
- gslfbot
- Gwene
- Harvest
- HawaiiBot
- humanlinks
- hyscore.io
- IAS crawler
- ICBot
- ICC-Crawler
- ichiro
- imrbot
- IndeedBot
- INETDEX-BOT
- InfoNaviRobot
- infoobot
- infoseek
- integromedb
- intelium_bot
- InterfaxScanBot
- ip-web-crawler.com
- IRLbot
- Iron33
- iskanie
- IsraBot
- istellabot
- it2media-domain-crawler
- James BOT
- JamesBOT
- Jamie's Spider
- JenkersBot
- JennyBot
- Jetbot
- Jetty
- JikeSpider
- JobboerseBot
- Jooblebot
- jpg-newsbot
- jyxobot
- k2spider
- K7MLWCBot
- kbcrawl
- Kemvibot
- Kenjin Spider
- keys-so-bot
- Keyword Density
- Knowings
- KomodiaBot
- KosmioBot
- Landau-Media-Spider
- larbin
- Laserlikebot
- lb-spider
- leadbox
- Leikibot
- LexiBot
- libWeb
- Linespider
- Linguee Bot
- linkapediabot
- LinkArchiver
- LinkCheck by Siteimprove.com
- linkdex
- LinkextractorPro
- LinkisBot
- linko
- LinkpadBot
- LinkScan
- lipperhey
- LivelapBot
- lkxscan
- LNSpiderguy
- lssbot
- lssrocketcrawler
- ltx71
- Luminator-robots
- lwp-trivial
- MaCoCu
- mappydata
- Mata Hari
- MauiBot
- MBCrawler
- MegaIndex
- MegaIndex.ru
- Meltawer
- Meltwater
- MeltwaterNews
- memorybot
- mention
- MetaJobBot
- MetaURI
- MIIxpc
- mindUpBot
- minicrawler
- Mister PiX
- MixnodeCache
- mlbot
- moatbot
- moget
- Mojeek
- MoodleBot
- Moreover
- MS Search 4.0 Robot
- MS Search 6.0 Robot
- MSIECrawler
- msrbot
- MTRobot
- Multiviewbot
- mytwip
- NAVER Blog Rssbot
- NaverBot
- Neevabot
- NerdByNature.Bot
- nerdybot
- NetAnts
- netEstate NE Crawler
- Neticle Crawler
- NetMechanic
- netresearchserver
- NetSystemsResearch
- newsharecounts
- NewsNow
- Newzbin
- NextGenSearchBot
- NICErsPRO
- niki-bot
- NimbleCrawler
- Nimbostratus-Bot
- NINJA bot
- NIXStatsbot
- NLUX_IAHarvester
- Nmap Scripting Engine
- NPBot
- NTENTbot
- Nuzzel
- OdklBot
- officestorebot
- omgilibot
- Openbot
- Openfind
- Openfind data gatherer
- OpenGraphCheck
- OpenHoseBot
- opinion-tracker
- Oracle Ultra Search
- OrangeBot
- Orthogaffe
- outbrain
- OutclicksBot
- page2rss
- PagePeeker
- PageThing
- peer39_crawler
- PerMan
- Pingdom
- PiplBot
- postrank
- PR-CY.RU
- Primalbot
- PrivacyAwareBot
- ProPowerBot
- ProWebWalker
- proxem
- psbot
- Pulsepoint
- purebot
- QueryN Metasearch
- Qwam content intelligence
- Radiation Retriever 1.1
- RankActiveLinkBot
- RankFlex
- Refindbot
- RegionStuttgartBot
- RepoMonkey
- RepoMonkey Bait & Tackle
- RetrevoPageAnalyzer
- ReverseEngineeringBot
- RidderBot
- Riddler
- Rivva
- Robozilla
- rssbot
- RSSingBot
- RukiCrawler
- RuxitSynthetic
- RyteBot
- SafeDNSBot
- SafeSearch microdata crawler
- SBL-BOT
- score3
- ScoutJet
- scribdbot
- Scrubby
- search.marginalia.nu
- SearchAtlas
- SearchmetricsBot
- searchpreview
- seekbot
- Seekport Crawler
- Seekr
- seewithkids
- semanticbot
- sempi.tech
- SemrushBot-BM
- SemrushBot-SA
- sentibot
- SEOkicks-Robot
- seoscanners
- seostar.co
- SEOstats
- SimpleCrawler
- SimpleScraper
- Sindup
- sistrix crawler
- SiteBot
- sitecheck.internetseer.com
- siteexplorer.info
- Siteimprove
- Siteimprove.com
- SiteSnagger
- SiteSucker
- Slack-ImgProxy
- Slackbot
- Slurp
- SocialRankIOBot
- Sogou
- Sogou inst spider
- Sogou spider2
- Sonic
- Sosospider
- SpankBot
- spanner
- spbot
- Spinn3r
- spotter
- SputnikBot
- Storebot-Google
- StorygizeBot
- StractBot
- Streamline3Bot
- SummalyBot
- summify
- SuperBot
- SurveyBot
- suzuran
- Swiftbot
- SWIMGBot
- Synthesio
- Sysomos
- Szukacz
- Taboolabot
- tagoobot
- Talkwater
- TangibleeBot
- Teleport
- TeleportPro
- Telesoft
- The Intraformant
- TheNomad
- theoldreader.com
- Thinklab
- tigerbot
- Titan
- toCrawl
- TombaPublicWebCrawler
- toplistbot
- ToutiaoSpider
- Traackr.com
- tracemyfile
- trafilatura
- trendeo
- trendkite-akashic-crawler
- trendybuzz
- trovitBot
- True_Robot
- TruliaBot
- turingos
- tweetedtimes
- twengabot
- Twurly
- UbiCrawler
- um-IC
- Updownerbot
- Upflow
- Uptime-Kuma
- Uptimebot.org
- UptimeRobot
- URL Control
- URL_Spider_Pro
- urlappendbot
- URLy Warning
- usasearch
- UsineNouvelleCrawler
- UT-Dorkbot
- Validator.nu
- VCI
- VCI WebViewer VCI WebViewer Win32
- vebidoobot
- vecteurplus
- Veoozbot
- verticalsearch
- Vigil
- VKRobot
- voilabot
- voltron
- VoluumDSP-content-bot
- vsw
- vuhuvBot
- W3C_I18n-Checker
- W3C_Unicorn
- W3C-checklink
- W3C-mobileOK
- WASALive-Bot
- wbsearchbot
- Web Image Collector
- web-archive-net.com.bot
- WebAuto
- WebBandit
- WebCapture 2.0
- webcompanycrawler
- WebCopier
- WebCopier v.2.2
- WebCopier v3.2a
- WebDataStats
- WebEnhancer
- WebmasterWorldForumBot
- webmon
- WebReaper
- WebSauger
- Website Quester
- WebStripper
- WebZIP
- winello
- WinHTTrack
- WiseGuys Robot
- wocbot
- woobot
- woorankreview
- WordupInfoSearch
- woriobot
- wotbox
- WWW-Collector-E
- WWW-Mechanize
- www.uptime.com
- Xenu
- Xenu Link Sleuth
- Xenu's
- Xenu's Link Sleuth 1.1c
- xovibot
- Yahoo Pipes 1.0
- YaK
- YandexMobileBot
- YandexVideo
- yanga
- Yellowbrandprotectionbot
- yoozBot
- YoudaoBot
- Youmag
- Zabbix
- Zao
- Zealbot
- zenback bot
- Zeus
- Zeus Link Scout
- zgrab
- Zite
- ZuperlistBot
- ZyBORG