300%. That's how much AI-driven bot traffic grew across the web in 2025, according to Akamai's State of the Internet report on AI and publishing. Akamai operates infrastructure that routes and protects a significant share of global internet traffic, so their data reflects actual network activity rather than survey estimates.
Publishers - news sites, blogs, and content platforms - absorbed the heaviest load.
Publishers Are Running Servers for AI Companies
AI bots are automated crawlers that collect web content to train machine learning models or to power AI-driven search tools. Unlike Googlebot, which indexes your site so humans can find it, AI training crawlers take your content and return nothing directly to you.
The publisher problem is financial: server costs, bandwidth, and CDN fees don't distinguish between a human reader and a bot. A 300% increase in bot traffic means a 300% increase in infrastructure load from visitors who generate zero ad revenue and zero subscriptions. For large publishers with enterprise contracts, this is a manageable line item. For small independent sites, it can be genuinely disruptive.
Akamai's framing in the report is notable - they use "botnet" language, suggesting some AI data collection runs in coordinated, high-volume campaigns rather than polite occasional indexing. That puts some operations in territory that looks a lot like infrastructure abuse, even if the intent isn't malicious.
The Options Are All Imperfect
Publishers have three real choices: block bots, absorb the costs, or license content. Each has problems.
Blocking via robots.txt - the file that tells crawlers which pages to avoid - works only if bots respect it. Several AI companies have been documented ignoring robots.txt instructions.
Rate limiting and bot detection add friction, but sophisticated crawlers route through residential IP addresses that look identical to human traffic. CAPTCHA systems create genuine barriers for real users while sophisticated bots route around them.
Licensing deals exist - some major AI companies pay publishers for access to their archives - but they're not available to small or independent publishers who have no leverage to negotiate the terms.
2025 was also the year AI search tools like Perplexity, ChatGPT search, and Google AI Overviews went mainstream. Each depends on crawling or licensing web content. The 300% surge in bot traffic tracks directly with that growth curve.
For content creators and small publishers who haven't checked their server logs recently, this is worth doing. AI bot traffic shows up as page views in analytics but doesn't behave like human traffic - high volume, zero session depth, unusual referrer patterns. If your bandwidth costs have climbed over the past 18 months without a matching rise in actual audience, bots are a likely contributor.