Show HN: 1,011 AI crawler requests. Google Analytics saw zero

Original Article Summary
Google Analytics can't see GPTBot or ClaudeBot. Here's how I built a server-side tracker that can — and what I found in 72 hours. Comments URL: https://news.ycombinator.com/item?id=47386037 Points: 1 # Comments: 0
Read full article at Adwait.me✨Our Analysis
Adwait's development of a server-side tracker to detect AI crawler requests, such as those from GPTBot or ClaudeBot, which are not visible in Google Analytics, highlights a significant blind spot in website analytics. This means that website owners may be underestimating the amount of AI-generated traffic to their sites, as traditional analytics tools like Google Analytics are not equipped to track these requests. The fact that Adwait's server-side tracker detected 1,011 AI crawler requests in just 72 hours, while Google Analytics saw zero, underscores the magnitude of this issue. To address this issue, website owners can take several steps: firstly, consider implementing a server-side tracker like Adwait's to get a more accurate picture of AI-generated traffic; secondly, review and update their llms.txt files to reflect the growing number of AI crawlers like GPTBot and ClaudeBot; and thirdly, monitor their website's traffic patterns and adjust their content strategies accordingly to account for the increasing presence of AI bots.
Related Topics
Track AI Bots on Your Website
See which AI crawlers like ChatGPT, Claude, and Gemini are visiting your site. Get real-time analytics and actionable insights.
Start Tracking Free →

