Specialist Search Engine
Specialised search engines have crawlers that scrape data for a particular vertical market or sector. Typical applications are for company data monitoring, for example amongst SME companies, government sectors, and other vertical markets. Typically, these are very niche players, and won't add much if any traffic, unless in a very niche sector, where they can add value.
Vendor
Bot Service
Recommendation
Description
You
You
Recommended
Not recommended
AI Search Engine - powered by You. AI/ Chat search
Wikimedia Foundation
Wikipedia Crawler
Recommended
Not recommended
It's Wikipedia! The COIBot is a bot that tries to track edits that are made by users who may have a conflict of interest.
Pipl
PIPL
Recommended
Not recommended
Just when you thought recruitment agents couldn't be more annoying, how about recruiters that use bots to crawl your domains for personal data including location and telephone numbers as well as names and job-titles? PiplBot crawler collects documents from the Web to build a searchable index which powers its People Search database. As part of the crawling, PiplBot does respect robots.txt so you can choose to exclude pages whose content you do not want included in Pipl Search.
Majestic
Majestic
Recommended
Not recommended
Majestic is a UK based specialist link intelligence whose platform spreads across 13 languages and over 60 countries to paint a map of the Internet independent of the consumer based search engines. MJ12Bot does not currently cache web content or personal data. Instead it maps the link relationships between websites to build a search engine. Used by many SEO companies to see the underlying patterns of linking strategies. This data is available to technologies and the public, either by searching for a keyword or a website at Majestic. MJ12Bot obeys Robots.txt and Crawl Delay.Because of the way the Majestic Search Engine builds its data the crawler is launched from a varied and distributed locations, indeed users of the service can invoke the crawler locally. Choose this one with care.
Hiya
Mr. Number.
Recommended
Not recommended
Searches for telephone numbers and has android & iphone apps to block unwanted calls. Uses a bot to search for the originating telephone numbers.
DueDil
Electricmonk
Recommended
Not recommended
Due Diligence tool aimed at tracking SME company records and the people that run them. Has an API and web platform for bulk searches.
Dassault Systèmes (3DS)
Exalead Thumbnails
Recommended
Not recommended
Exalead® is a leading search-based application platform provider to business and government. Exalead's worldwide client base includes leading companies such as PricewaterhouseCooper, ViaMichelin, GEFCO, WorldBank and Sanofi Pasteur, and more than 100 million unique users a month use Exalead's technology for search.
Dassault Systèmes (3DS)
Exalead
Recommended
Not recommended
Exalead® is the leading search-based application platform provider to business and government. Exalead's worldwide client base includes leading companies such as PricewaterhouseCooper, ViaMichelin, GEFCO, WorldBank and Sanofi Pasteur, and more than 100 million unique users a month use Exalead's technology for search.
Dassault Systèmes (3DS)
CloudView
Recommended
Not recommended
Exalead is reshaping the digital content landscape with its platform, Exalead CloudView™, which uses advanced semantic technologies to bring structure, meaning and accessibility to unused or under-used data in the new hybrid enterprise and Web information cloud. CloudView collects data from virtually any source, in any format, and transforms it into structured, pervasive, contextualized building blocks of business information.It will crawl your site to gather and add content to the Exalead business and government search engine.
Cliqz
Cliqz
Recommended
Not recommended
Cliqz is a European based, independent, provider of browser extensions allowing users to perform web searches. Like traditional search engines they crawl sites to index content. The Cliqzbot crawler respects Robots.txt
Business Seek
BusinessSeek.biz_Spider
Recommended
Not recommended
Business Seeker is a business directory site that scrapes business contacts from other sites to host on its directory.
aiHitdata
Company Data Search
Recommended
Not recommended
Specialist search engine for company data monitoring. Hitdata claims to use artificial intelligence/machine learning, to automated the gathering and building up of detailed company information from the web using this bot. Respects robots.txt.
AddSearch
AddSearch
Recommended
Not recommended
AddSearch is a specialist search engine designed to be embedded in your own website, to allow relevant and tailored search results to be surfaced to your user base. If you use the AddSearch product you should allow this bot to prevent issues.