The Internet is for humans, not robots

The really good robots just can't seem to keep up with the growing number of humans and bad robots on the web. Good robots are the helpers, they're designed to crawl the web for archiving purposes, to populate RSS feeds and search engines, and to help with other automated tasks that assist humans. Bad bots, by contrast, are the impersonators, data thieves, and other rogue agents.

That was the main finding of a new report by the security firm, Imperva Incapsula, which is set to release its annual survey of Internet-bot traffic this week. Incapsula has been studying web traffic for four years. In results, bots had always driven the majority of web activity. But this year, there was a surprise. "We saw a changing of the guard, with humans stepping in to become the Internet's new majority," Incapsula wrote. In 2015, humans were responsible for nearly 52 percent of all online traffic. Two years ago, humans drove less than 39 percent of overall web traffic.

This apparent reversal turns out to be more nuanced than it appears at first. The volume of Internet activity by good bots, relative to humans and bad bots, declined - but did so in part because human online activity grew in 2015, and bad bot traffic stayed steady compared to last year. "Basically, what we had here was a case of good bots collectively not keeping up with growth of human and bad bot traffic," Incapsula wrote.

According to The Atlantic, on popular websites that get the most overall traffic from humans, the share of good-bot activity dropped significantly, from 22 percent to 9 percent. But among the sample of websites that logged fewer than 100,000 human visits a day, good bot traffic went up. "It appeared that, the more popular a website got, the harder it was for the good bots to keep up with the influx of human and bad bot visits," Incapsula wrote.

For one thing, good bots aren't enticed by the same factors that make a website popular to humans. "In fact, the more popular a website gets, the faster its human population tends to grow," Incapsula said. Although, it seems reasonable to expect a popular website would draw more scammer bots, "the amounts of extra sessions generated are negligible."

Qoshe says that, the backdrop for all this web activity is a population of human Internet users that's still growing dramatically. The number of people who used the Internet this year exceeded 3.1 billion, up from 2.9 billion last year. Individuals continue to spend longer periods of time using the Internet, too. "The compound effect of these growth trends, and the fact that only humans are motivated to consume and share popular content, are the reason why good bots-who lack the same motivations-simply can't keep up," Incapsula wrote. 

Real Time Analytics