David McWilliams: The economics of the AI boom are actually quite old-fashioned

Original Article Summary
Can all this credit continue to flow into a sector that is spending so much but – at least so far – is earning so little?
Read full article at The Irish Times✨Our Analysis
Irish Times' publication of David McWilliams' analysis that “the economics of the AI boom are actually quite old‑fashioned” highlights how massive credit inflows are fueling AI startups that are still burning cash faster than they generate revenue. For website owners, this surge in under‑profitable AI ventures means a likely spike in AI‑driven traffic—both legitimate bots crawling for data and experimental bots testing new models. As these bots proliferate, they can inflate analytics, skew SEO metrics, and increase server load, especially on sites that host AI‑related content or APIs. **Actionable tips:** 1. Update your llms.txt to explicitly list high‑volume AI crawlers (e.g., OpenAI‑Bot, Anthropic‑Crawler) and set appropriate crawl‑rate limits to protect bandwidth. 2. Deploy a bot‑detection layer (e.g., using request headers and behavior profiling) that flags traffic originating from newly funded AI startups, allowing you to route them to a lightweight “bot‑only” version of your site. 3. Regularly audit your server logs for abnormal request patterns tied to AI research labs and adjust your robots.txt or firewall rules to mitigate any abusive scraping.
Track AI Bots on Your Website
See which AI crawlers like ChatGPT, Claude, and Gemini are visiting your site. Get real-time analytics and actionable insights.
Start Tracking Free →


