LLMS Central - The Robots.txt for AI
Web Crawling

NY Times Accuses Brave Of Scraping And Exploiting Content via @sejournal, @martinibuster

Search Engine Journal••2 min read
Share:
NY Times Accuses Brave Of Scraping And Exploiting Content via @sejournal, @martinibuster

Original Article Summary

New York Times lawsuit accuses Brave of selling access to paywalled content to AI companies and depriving publishers of ad income. The post NY Times Accuses Brave Of Scraping And Exploiting Content appeared first on Search Engine Journal.

Read full article at Search Engine Journal

✨Our Analysis

NY Times' lawsuit against Brave alleges the browser sold access to its paywalled articles to AI companies, effectively scraping and monetizing copyrighted content without publisher consent. For website owners, this case highlights a growing risk that AI providers may source premium, paywalled material through browser extensions or ad networks that bypass traditional paywalls. If your site relies on subscription revenue, undisclosed AI scraping can erode both ad impressions and direct subscriber value, as AI models repurpose your content for downstream services. The lawsuit also signals that regulators may soon enforce stricter disclosure and consent requirements for any entity that harvests site content for AI training. **Actionable tips:** 1. **Update your llms.txt** to explicitly deny AI crawlers from accessing paywalled or subscriber‑only pages (e.g., `User-agent: *\nDisallow: /paywall/`). 2. **Deploy bot‑detection scripts** that flag traffic originating from known AI data‑scraping services or Brave’s ad‑network domains, allowing you to block or throttle those requests. 3. **Monitor referral headers** for unexpected sources like `brave.com` or AI‑related subdomains, and set up alerts in your analytics platform to catch spikes that may indicate bulk content harvesting.

Related Topics

Search

Track AI Bots on Your Website

See which AI crawlers like ChatGPT, Claude, and Gemini are visiting your site. Get real-time analytics and actionable insights.

Start Tracking Free →