Human-eyes only: Bots now 'account for 61% of web traffic'

In some staggering research by Incapsula; the firm suggests that 61% of all website traffic is now generated by automated bots - a rise of an incredible 21% over last year.

Of course not all bot traffic is malicious; much of it such as search engine “spiders” is actually helpful and improve the web experience for users.

So do you want the good, or the better news first?

Most of the growth in traffic was in these “friendly” bots. Examples of these are; tools used by search engines to crawl websites in order to index their content, by analytics companies to provide feedback about how a site is performing, and to carry out other specific tasks - such as helping the Internet Archive preserve content.

The activity by these bots has grown by 55% over the year; which is thought to be fuelled primarily by search engines updating their content more regularly.

As for the arguably better news, “nasty” bots have been on a decline across (most of) the board.

Notably, there has been a 75% drop in the frequency spam links were being automatically posted; and also a 10% drop in hacking tool bot activities – such as use of code to distribute malware, to steal credit cards, and to hijack and deface websites.

But it’s not all good news, there has been an 8% rise in the use of “impersonator bots” which disguise themselves as from a search engine or other legitimate source to fool security measures. These forms of bots tend to be made with the aim to carry out a specific malicious activity; such as a DDoS (Distributed Denial of Service) attack to take a website or service offline.

Incapsula, who specialise in website acceleration and security, sourced the information from 20,000 sites operated by their clients and observed 1.45 billion bot visits over a 90 day period.

Dr Ian Brown, associate director at Oxford University's Cyber Security Centre, said: “There will also be some unavoidable fuzziness in their data, given that they are trying to measure malicious website visits where by definition the visitors are trying to disguise their origin.”

He also noted this major year-on-year increase in traffic will put website operators under more strain; but also states: “I think that, at this scale, it's something they can cope with.”

What do you think about Incapsula’s findings? Are they surprising or expected?

Related Stories

Leave a comment


This will only be used to quickly provide signup information and will not allow us to post to your account or appear on your timeline.