Automating Bot Verification

You blocked what? Verifying bots using weblogs and robots.txt is a painful and thankless task.

‍

At some point most website owners have had that sinking feeling - they have accidentally blocked a service your marketing team relies on. If you’re blocking a major search engine, the repercussions can be unforgiving. It's great the viral ad campaign took off, less great that your WAF engine rate limited the excessive traffic and blocked the majority of additional human visitors that your marketing team had attracted to your website. The WAF rules did their job - but may have threatened yours.

‍

Complaining that you had no notice of the marketing activity will fall on deaf ears. The website is simply expected to respond to all types of traffic regardless. The job of blocking millions of bad bots and protecting the web site often goes unrecognized. In contrast, blocking a legitimate service can be career limiting.

‍

Faced with the inevitable complaints from the marketing department, Webmasters are understandably reluctant to block the common user agents. The cybercriminals know this and take maximum advantage of how easy it is to spoof common user agents. If the cybercriminals hit the jackpot and get lucky, the webmaster will actually whitelist the disguised bot, giving it privileged access to the entire estate. Whitelists are rarely checked and maintained, so this bot could remain undetected for a while.

‍

How can VerifiedVisitors save you?

VerifiedVisitors neatly and elegantly takes this headache away for you. We operate a subscription-based whitelist, which automatically verifies the legitimate bots/vendors, so you don’t have to. We do the heavy-lifting of verifying all the common marketing and web tools that you need. You simply use our verified bot database and select the services you require, leaving us to do the rest. As the user agents or IP addresses change, our service is updated and sends you the latest definitions automatically to ensure your whitelist is verified and accurate at all times. You can even share the VerifiedVisitors portal with the marketing team, who can simply check the services they use. Problem solved!

Taking Care of the Good Guys

Now that you’ve taken care of the legitimate bots you want on your website, sorting out the bad bots just became a whole lot easier. Any bots masquerading as the legitimate agents can be blocked because you already know the desired bot has been checked and verified. Using VerifiedWhiteList there is less chance of the cybercriminals using an existing agent and spoofing it to gain unauthorized access to your site. Now that you know you can’t do any harm by blocking legitimate bots, you can use your existing security tools to manage the rest.

‍