Designing for AI deep research: lessons from Google, Perplexity and Microsoft

Google's design guidelines for AI deep research, highlighted alongside Perplexity’s conversational search flow and Microsoft’s Copilot‑style interfaces, emphasize six UX questions that shape how users conduct time‑intensive, multi‑step queries. For website owners, this focus on deep‑research AI agents means a surge in automated traffic that mimics human exploratory behavior—bots will issue sequential, context‑rich requests, follow links, and scrape content to build comprehensive answers. Unlike simple single‑turn queries, these agents can generate high‑frequency, low‑bounce sessions that inflate analytics and potentially strain server resources. Moreover, the nuanced intent detection these platforms employ can bypass traditional bot filters, making it harder to distinguish genuine user engagement from AI‑driven research crawls. **Actionable tips:** 1. **Update your llms.txt** to explicitly list permitted AI agents (e.g., “Google‑AI‑Research”, “Perplexity‑Bot”) and define crawl rate limits for deep‑research bots, preventing them from overwhelming your site. 2. **Implement session‑pattern detection** by tracking sequential request chains (same IP, increasing depth, similar query semantics) and flagging them for review or throttling via your server’s rate‑limiting rules. 3. **Leverage bot‑traffic analytics** in your dashboard to separate deep‑research AI sessions from regular users, enabling precise reporting and targeted adjustments to your llms.txt policies.