LLMS Central - The Robots.txt for AI

simplescraper.io

Last updated: 10/30/2026valid

Independent Directory - Important Information

This llms.txt file was publicly accessible and retrieved from simplescraper.io. LLMS Central does not claim ownership of this content and hosts it for informational purposes only to help AI systems discover and respect website policies.

This listing is not an endorsement by simplescraper.io and they have not sponsored this page. We are an independent directory service with no affiliation to the listed domain.

Copyright & Terms: Users should respect the original terms of service of simplescraper.io. If you believe there is a copyright or terms of service violation, please contact us at support@llmscentral.com for prompt removal. Domain owners can also claim their listing.

Current llms.txt Content

# Simplescraper

> No-code web scraping by dashboard and free Chrome extension, with AI extraction and website change monitoring. Turn any site into structured data or an API.

Connect AI tools like Claude, Cursor, and ChatGPT to Simplescraper via MCP. Manage recipes and scrape websites directly from your AI assistant. Complete Simplescraper API reference: authentication, scraping endpoints, recipe management, Smart Extract, and webhook configuration.

## For AI agents and developers

- [MCP server for Claude, Cursor, ChatGPT and other AI agents](https://simplescraper.io/docs/mcp-server): endpoint https://mcp.simplescraper.io/mcp. Connect AI tools like Claude, Cursor, and ChatGPT to Simplescraper via MCP. Manage recipes and scrape websites directly from your AI assistant.
- [REST API](https://simplescraper.io/docs/api-guide): base URL https://api.simplescraper.io/v1. Complete Simplescraper API reference: authentication, scraping endpoints, recipe management, Smart Extract, and webhook configuration.
- [Screenshot API](https://simplescraper.io/docs/screenshot-api): Capture full-page or viewport screenshots of any URL via the Simplescraper Screenshot API. Supports PNG, JPEG, and WebP with configurable viewport and page-load options.
- [Google Search API](https://simplescraper.io/docs/google-search-api): Get Google search results as structured JSON from one API call: up to 100 organic results with position, title, URL and snippet, plus the answer box, knowledge graph, People also ask and related searches. 2 credits per page of ten results.
- [URL Extractor API](https://simplescraper.io/docs/extract-urls-api): Extract all URLs from any website's sitemap via the Simplescraper URL Extractor API. Discover sitemaps automatically from common paths and robots.txt.
- [Smart Extract](https://simplescraper.io/docs/smart-data-extract): Tell Simplescraper's AI what data you want and it finds and extracts it automatically. Returns reusable CSS selectors for reliable, repeated scraping.
- [Enhance scraped data using AI](https://simplescraper.io/docs/enhance-scraped-data-using-ai): Turn scraped data into the information you need. Simplescraper AI Enhance summarizes text, pulls out details like emails, answers yes or no questions and sorts rows into categories.
- [Extracting websites to Markdown](https://simplescraper.io/docs/extract-markdown): Convert any webpage or entire website into clean Markdown format with Simplescraper. Ideal for feeding web content into AI models like ChatGPT and Claude.

## Getting started

- [Pricing](https://simplescraper.io/#pricing)
- [Chrome extension](https://chrome.google.com/webstore/detail/lnddbhdmiciimpkbilgpklcglkdegdkg)
- [All features](https://simplescraper.io/features)
- [Overview](https://simplescraper.io/docs): Get started with Simplescraper: extract data from any website as CSV, JSON or Markdown, and monitor web pages for changes with email and webhook alerts.
- [Choosing how to scrape](https://simplescraper.io/docs/local-scraping-cloud-scraping-difference): Compare browser, cloud, and API scraping in Simplescraper. Learn which method suits your needs for speed, scale, scheduling, and automation.
- [Credits](https://simplescraper.io/docs/credits): Understand how Simplescraper credits work: usage rates for browser scraping, cloud scraping and AI features. Credits renew monthly on paid plans.
- [FAQ](https://simplescraper.io/docs/faq): Answers to common Simplescraper questions: how much data you can scrape, credits, logins, recipes, data retention, proxies and rate limits.

## Features

- [Perfectly Screenshot Any Website in Seconds](https://simplescraper.io/features/screenshot-api): Capture pixel-perfect screenshots via API or bulk-capture from the dashboard. Control viewport, format, full-page rendering, and more.
- [AI-Powered Web Scraping](https://simplescraper.io/features/ai-extraction-and-enhance): Use AI to automatically extract structured data from any website and enhance it with summaries, translations, and analysis. No CSS selectors needed.
- [Scrape Websites Behind a Login](https://simplescraper.io/features/scrape-behind-login): Extract data from sites requiring login with cookies or credentials. Works with LinkedIn, Facebook, Instagram, and any password-protected site.
- [Bulk-scrape a list of URLs into structured data with no code](https://simplescraper.io/features/scrape-list-of-urls): Select the data you want, paste up to 10,000 URLs and get structured data for every page. Download CSV or JSON or send it anywhere with a webhook. No code.
- [Website Change Monitoring](https://simplescraper.io/features/website-change-monitoring): Monitor a web page, an area or one element for changes and get an email, webhook or agent alert showing exactly what changed.
- [Scheduled Web Scraping](https://simplescraper.io/features/schedule-and-monitor): Run scrapers every 30 minutes, hourly, daily or on a custom schedule in your timezone, and send every run to Google Sheets, a webhook or the API.
- [Scrape Data Directly to Google Sheets](https://simplescraper.io/features/google-sheets-integration): Send scraped data automatically to Google Sheets. Real-time sync, scheduled updates, and seamless workflow integration.
- [Turn Any Website Into Structured Data](https://simplescraper.io/features/extract-structured-data): Automatically detect and extract tables, lists, product info, and structured data from any website without writing selectors.
- [Extract All Links from Any Website](https://simplescraper.io/features/url-extractor): Find and extract all URLs from websites. Perfect for building crawl lists, finding resources, or mapping site structures.
- [Remove Duplicate Scraped Data Automatically](https://simplescraper.io/features/deduplication): Eliminate duplicate entries in your scraped data. Smart deduplication based on custom fields ensures clean datasets.
- [Send Scraped Data Anywhere With Webhooks & Zapier](https://simplescraper.io/features/webhooks-integration): Send scraped data to any app using webhooks. Integrate with Zapier and Make integrations for automated workflows.
- [AI Agent Automation (MCP)](https://simplescraper.io/features/ai-agent-automation-mcp-server): Connect Simplescraper to AI agents via MCP protocol. Enable Claude, Cursor, and ChatGPT to scrape live data for enhanced workflows.
- [Turn Websites into Live APIs](https://simplescraper.io/features/realtime-api-endpoints): Create instant APIs from any website. Get fresh data on-demand with our powerful REST API endpoints.
- [Scrape Multi-Page & Infinite Scroll Sites](https://simplescraper.io/features/pagination-and-infinite-scroll): Automatically handle pagination, load more buttons, and infinite scroll. Scrape entire catalogs and feeds effortlessly.

## Docs

- [Overview](https://simplescraper.io/docs): Get started with Simplescraper: extract data from any website as CSV, JSON or Markdown, and monitor web pages for changes with email and webhook alerts.
- [Choosing how to scrape](https://simplescraper.io/docs/local-scraping-cloud-scraping-difference): Compare browser, cloud, and API scraping in Simplescraper. Learn which method suits your needs for speed, scale, scheduling, and automation.
- [Credits](https://simplescraper.io/docs/credits): Understand how Simplescraper credits work: usage rates for browser scraping, cloud scraping and AI features. Credits renew monthly on paid plans.
- [FAQ](https://simplescraper.io/docs/faq): Answers to common Simplescraper questions: how much data you can scrape, credits, logins, recipes, data retention, proxies and rate limits.
- [What is browser scraping](https://simplescraper.io/docs/browser-scraping-guide): Learn browser scraping with Simplescraper: get instant results from the Chrome extension side panel, scrape behind logins, and export data.
- [Scraping and saving recipes](https://simplescraper.io/docs/how-scrape-save-recipes): Scraping with Simplescraper 3.0: it finds the list you want, names the columns with AI and works out the next page for you, so you go from a web page to clean data in a few clicks.
- [Expanding list selection](https://simplescraper.io/docs/expand-list-selection): Scrape every item on a page when the list is split into sections, such as jobs grouped by department: 'Include similar' widens the selection to every matching list.
- [AI field naming](https://simplescraper.io/docs/ai-property-naming): Rename fields like sc-1x2y3z to clear names like job_title and price in one click with Simplescraper's 'AI rename', which reads the values in each field.
- [Bulk scraping in the browser](https://simplescraper.io/docs/browser-batch-scraping): Run the same fields over a list of up to 10,000 URLs from your browser: paste them, upload a CSV file or import the links an earlier scrape collected.
- [Sub-page (deep) scraping](https://simplescraper.io/docs/browser-subpage-scraping): Sub-page (deep) scraping in the browser: scrape a list of links, then the same fields from every page they lead to, with no saved recipe.
- [What is cloud scraping](https://simplescraper.io/docs/dashboard-guide): Guide to the Simplescraper cloud dashboard: run recipes in the background, set schedules, bulk scrape URLs, and configure integrations.
- [Multiple page and infinite scroll](https://simplescraper.io/docs/multiple-page-infinite-scroll): Scrape every page of a list, not just the first: Simplescraper finds the next page button, load more button or infinite scroll and scrapes as many pages as you choose, in the extension and in the cloud.
- [Bulk scraping in the cloud](https://simplescraper.io/docs/crawling-lists-of-urls): Scrape up to 10,000 URLs in parallel using Simplescraper's cloud bulk scraper. Generate URL lists, import from previous scrapes, and run in the background.
- [Sub-page (deep) scraping](https://simplescraper.io/docs/deep-scraping-urls): Sub-page (deep) scraping in the cloud: one recipe scrapes a list of links, a second scrapes the details from each linked page. Ideal for job boards, product catalogs and listings.
- [Scheduling your webscraping](https://simplescraper.io/docs/scheduling-webscraping): Automate your scraping with Simplescraper's scheduler. Set intervals from every 2 minutes to specific days and times with timezone support.
- [Datasets (incremental scraping)](https://simplescraper.io/docs/incremental-scraping): Datasets (incremental scraping) let you scrape the same pages again and again and save only the new data: every run adds its new rows to one growing dataset, with no duplicates.
- [Scraping data behind a login](https://simplescraper.io/docs/scrape-behind-login): Scrape password-protected websites with Simplescraper using saved credentials or browser cookies. Supports multi-factor and SSO logins.
- [Data retention](https://simplescraper.io/docs/data-retention): How long scraped data is stored in Simplescraper: results from each run are available for 30 days, incremental datasets persist and grow, and data sent to integrations is yours to keep.
- [Ready-made recipes](https://simplescraper.io/docs/ready-made-recipes): Browse Simplescraper's library of ready-made scrape recipes for Google, LinkedIn, Reddit, Instagram, Yelp, and other popular websites.
- [Monitor a web page for changes](https://simplescraper.io/docs/monitor-web-page-changes): Monitor any website for changes. Simplescraper checks the text or appearance of a web page on your schedule and alerts you by email, webhook or AI agent when it changes.
- [Extract API v1](https://simplescraper.io/docs/api-guide): Complete Simplescraper API reference: authentication, scraping endpoints, recipe management, Smart Extract, and webhook configuration.
- [Screenshot API](https://simplescraper.io/docs/screenshot-api): Capture full-page or viewport screenshots of any URL via the Simplescraper Screenshot API. Supports PNG, JPEG, and WebP with configurable viewport and page-load options.
- [Google Search API](https://simplescraper.io/docs/google-search-api): Get Google search results as structured JSON from one API call: up to 100 organic results with position, title, URL and snippet, plus the answer box, knowledge graph, People also ask and related searches. 2 credits per page of ten results.
- [URL Extractor API](https://simplescraper.io/docs/extract-urls-api): Extract all URLs from any website's sitemap via the Simplescraper URL Extractor API. Discover sitemaps automatically from common paths and robots.txt.
- [Smart Extract](https://simplescraper.io/docs/smart-data-extract): Tell Simplescraper's AI what data you want and it finds and extracts it automatically. Returns reusable CSS selectors for reliable, repeated scraping.
- [Enhance scraped data using AI](https://simplescraper.io/docs/enhance-scraped-data-using-ai): Turn scraped data into the information you need. Simplescraper AI Enhance summarizes text, pulls out details like emails, answers yes or no questions and sorts rows into categories.
- [Extracting websites to Markdown](https://simplescraper.io/docs/extract-markdown): Convert any webpage or entire website into clean Markdown format with Simplescraper. Ideal for feeding web content into AI models like ChatGPT and Claude.
- [MCP server](https://simplescraper.io/docs/mcp-server): Connect AI tools like Claude, Cursor, and ChatGPT to Simplescraper via MCP. Manage recipes and scrape websites directly from your AI assistant.
- [Scraping data into Google Sheets](https://simplescraper.io/docs/saving-data-google-sheets): Connect Simplescraper to Google Sheets and automatically save scraped data to a spreadsheet every time a recipe runs. Append or replace modes available.
- [Scraping data into Airtable](https://simplescraper.io/docs/scraping-data-into-airtable): Send scraped data directly to Airtable bases using Personal Access Tokens. Configure field mapping and auto-sync results from any Simplescraper recipe.
- [Scraping data into Zapier](https://simplescraper.io/docs/scraping-data-into-zapier): Connect Simplescraper to Zapier via webhooks and send scraped data to 2,000+ apps. Step-by-step setup guide with webhook configuration.
- [Scraping data into Make](https://simplescraper.io/docs/scraping-data-into-integromat): Send scraped website data to Make (formerly Integromat) scenarios via webhooks. Setup takes under a minute with Simplescraper's built-in integration.
- [Scraping data into your email inbox](https://simplescraper.io/docs/scraping-data-into-email): Receive scraped data directly in your email inbox as a preview, CSV attachment, or both. Configure from the Send tab of any recipe.
- [Sending data to a webhook](https://simplescraper.io/docs/sending-data-to-webhook): Send scraped data to any URL as a JSON webhook. Receive structured data in your own backend, automation tool, or any service that accepts HTTP POST requests.
- [Common problems](https://simplescraper.io/docs/troubleshooting): Fix common Simplescraper issues: timeout errors, missing data, login failures, and incorrect results. Includes resolution steps for each scenario.
- [Scrape recipe health](https://simplescraper.io/docs/recipe-health): Monitor the health of your scraping recipes with automated checks. Spot issues before they affect your data, and use Site Unblocker to get through bot protection.
- [Adding a password](https://simplescraper.io/docs/adding-password-login): Add a password login to your Simplescraper account alongside Google authentication. Useful for sharing access or using alternative login methods.
- [Your subscription](https://simplescraper.io/docs/managing-subscription): Manage your Simplescraper subscription: upgrade, downgrade, add credit topups, update payment details, or cancel from your account page.
- [Delete account](https://simplescraper.io/docs/delete-account): Delete your Simplescraper account from the My account page. The action is irreversible and cancels any active subscription.

## Free tools

- [Free URL Extractor — Extract All Links from Any Website Instantly](https://simplescraper.io/extracturls)
- [Free Website to Markdown, JSON, CSV Converter for LLMs](https://simplescraper.io/extractmarkdown)

## Optional

- [Blog](https://simplescraper.io/blog)
- [Scrape templates](https://simplescraper.io/templates)
- [Developer guides](https://simplescraper.io/learn)
- [Privacy](https://simplescraper.io/docs/privacy)
- [Terms](https://simplescraper.io/docs/terms-and-conditions)
- [Full documentation as one file](https://simplescraper.io/llms-full.txt)

Version History

Version 110/30/2026, 9:02:42 PMvalid
16468 bytes

Categories

blogdocumentationdocshealth

Visit Website

Explore the original website and see their AI training policy in action.

Visit simplescraper.io

Content Types

pagesapidocumentationguides

Recent Access

No recent access

API Access

Canonical URL:
https://llmscentral.com/simplescraper.io/llms.txt
API Endpoint:
/api/llms?domain=simplescraper.io
simplescraper.io - llms.txt File | AI Training Guidelines | LLMS Central