Bots or Spiders are basically software applications that run automated computer programs (scripts) over the internet that are hitting your website. They do it for various reasons.
Typically, bots or spiders execute tasks that are both simple and essentially repetitive, at a much higher rate than would be likely for a human alone. Sometimes it’s a search engine looking to list your content on their website. Sometimes it’s a program looking to see if your blog has presented new content so that it can inform someone in their news reader. Sometimes it’s a service that an organization or individual has hired to make sure that their server is up and running, that its loading speed is normal, etc.
Bot traffic may impact your Analytics report. There are two main types of Bot that impact your Google Analytics data.
1) Zombie bots fully render your website. They produce analytics spam by triggering your analytics code as a consequence of their activity.
2) Ghost bots never actually visit your site. They are nothing more than nuisance spam and show up under referral traffic in the Google Analytics report.
How do you detect bot traffic in GA?
There are some bots you don’t need to worry about. Good bots, such as search engine crawlers, are normally excluded from Google Analytics data by default. These bots will also respect the warnings in your robots.txt file and, usually speaking, won’t exploit your data.
Make sure to check your GA dashboard regularly; you could encounter an unexpected spike in traffic on some occasions. If there were no special promotions, campaigns or social events that could justify the sudden increase in traffic, then it is likely due to bot or spider traffic.
Some initial indications of bot or spider traffic in your Google Analytics reports can be:
- High bounce rates
- Low average session duration
- No goal completion
- High values for new visitors
- Specific browser receives high traffic
How to Exclude Bot Traffic
There are couple of ways by which you can remove bot traffic from Google Analytics; a suggested course of action may go something like this:
1) Check the “Exclude all hits from known bots and spiders” option in the Admin View settings. It’s a very helpful and easy to fix, but it’s not perfect.2) Apply filter by hostname. You will need to set up a filter by Hostname and take note of all the valid hostnames of your website (sub domain or main domain)3) Apply filter to exclude the bot domain. You can find lists of all the bot domains from the internet; make a regex list and exclude them from view.Recommended spam blocker tools:
The tool automatically applies and updates Google Analytics filters that exclude spam references or ghost traffic from your Analytics reports.
It’s a great start to solve all your smart bot and spider problems by following the best practices above, and hopefully enjoy and make good decisions based on your clean data.