โ๏ธ Cloudflare Worker Setup Guide
Track AI bots on ANY website - WordPress, Shopify, Wix, static sites, and more. No code changes needed on your actual site!
Universal Solution
Works for ANY platform - WordPress, Shopify, Wix, Squarespace, static sites, custom apps
10-15 Min Setup
Simple step-by-step process. No coding required. Free Cloudflare account.
Server-Side Tracking
Catches ALL AI bots (GPTBot, Claude, etc.) that don't execute JavaScript
Sign Up for Cloudflare (Free)
- 1. Go to: https://dash.cloudflare.com/sign-up
- 2. Create a free account
- 3. Verify your email
๐ฐ Cost: FREE (Free plan is sufficient for bot tracking)
Add Your Website to Cloudflare
- 1. Click "Add a Site" in Cloudflare dashboard
- 2. Enter your domain (e.g.,
example.com) - 3. Click "Add site"
- 4. Select "Free" plan
- 5. Click "Continue"
Cloudflare will scan your DNS records (takes ~1 minute).
โ ๏ธ Verify DNS Records (IMPORTANT!)
Cloudflare auto-imports your DNS records, but you MUST verify them before switching nameservers!
- 1. After adding your site, Cloudflare shows your DNS Records
- 2. Review all records - Cloudflare usually gets 80-90% of them
- 3. Compare with your current DNS (check your domain registrar)
- 4. Add any missing records manually (click "Add record")
- 5. Pay attention to:
- โข A records (your website IP)
- โข CNAME records (subdomains, CDN)
- โข MX records (email - CRITICAL!)
- โข TXT records (verification, SPF, DKIM)
๐จ Missing records = broken functionality! If you skip this step, your email or website might break after switching nameservers.
๐ก Tip: Most registrars (GoDaddy, Namecheap) let you export DNS records. You can then import them into Cloudflare's DNS page to save time. Cloudflare also auto-scans and imports most records when you add your site.
Update Your Nameservers
Cloudflare will show you 2 nameservers like:
- 1. Go to your domain registrar (GoDaddy, Namecheap, etc.)
- 2. Find "Nameservers" or "DNS Settings"
- 3. Replace existing nameservers with Cloudflare's nameservers
- 4. Save changes
โฐ Wait 5-60 minutes for DNS to propagate. Cloudflare will email you when it's active.
Create the Worker
- 1. In your domain's Cloudflare dashboard, click "Workers Routes" (left sidebar)
- 2. Click "Manage Workers" button (top right)
- 3. Click "Get started" on "Start with Hello World!"
- 4. Name your worker:
llms-central-bot-tracker - 5. Click "Deploy"
๐ก Tip: The "Hello World" template creates a basic worker that you'll customize in the next step.
Add the Tracking Code
- 1. Click "Edit Code" button
- 2. Delete all existing code in the editor
- 3. Copy the code below and paste it
- 4. Click "Deploy"
/**
* LLMS Central AI Bot Tracker - Cloudflare Worker
*
* This worker intercepts all requests to your website and tracks AI bot visits
* before forwarding the request to your origin server.
*/
// AI bot patterns to detect
const AI_BOT_PATTERNS = [
'gptbot', 'chatgpt', 'claude', 'anthropic', 'google-extended',
'googlebot', 'bingbot', 'applebot', 'facebookexternalhit',
'meta-externalagent', 'perplexitybot', 'youbot', 'bytespider',
'diffbot', 'grokbot', 'x-bot', 'grok', 'x.ai', 'xai',
'cohere', 'amazonbot', 'yandexbot', 'baiduspider'
];
const LLMS_CENTRAL_API = 'https://llmscentral.com/api/bot-tracker';
export default {
async fetch(request, env, ctx) {
const userAgent = request.headers.get('user-agent') || '';
const userAgentLower = userAgent.toLowerCase();
const isBot = AI_BOT_PATTERNS.some(pattern => userAgentLower.includes(pattern));
if (isBot) {
const domain = env.DOMAIN || new URL(request.url).hostname;
const page = new URL(request.url).pathname;
const clientIP = request.headers.get('cf-connecting-ip') || 'Unknown';
// Fire and forget - track bot visit
ctx.waitUntil(
fetch(LLMS_CENTRAL_API, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'User-Agent': userAgent,
'X-Forwarded-For': clientIP
},
body: JSON.stringify({ domain, page })
}).catch(err => console.error('Tracking error:', err))
);
}
return fetch(request);
}
};Configure Environment Variable
- 1. Go to Settings tab
- 2. Click "Variables"
- 3. Under "Variables and Secrets", click "Add"
- 4. Type: Keep as
Text - 5. Variable name:
DOMAIN - 6. Value: Your domain (e.g.,
example.com) - 7. Click "Save" or "Add variable"
Add Worker Route
- 1. Go back to your website in Cloudflare (not Workers section)
- 2. Click on your domain
- 3. Go to "Workers Routes" (under Workers & Pages)
- 4. Click "Add route"
- 5. Route:
*example.com/*(replace with your domain) - 6. Worker: Select
llms-central-bot-tracker - 7. Click "Save"
โ ๏ธ Disable "Block AI Bots" (CRITICAL!)
If you don't do this step, the worker won't be able to track AI bots!
- 1. In your domain's Cloudflare dashboard, click "Security" (left sidebar)
- 2. Click "Settings"
- 3. Find "Block AI bots" section
- 4. Click the edit icon (pencil) next to "Block AI Bots scope"
- 5. Click the "Do Not Block (allow crawlers)" feature
- 6. Click "Save"
๐ก Why? Cloudflare's "Block AI bots" feature blocks bots BEFORE the worker can track them. You need to disable it so the worker can intercept and track bot visits first.
โ Test Your Installation
Quick Test:
- 1. Visit your website normally - it should work exactly as before
- 2. Go to: https://llmscentral.com/dashboard?tab=bots
- 3. Look for your domain
- 4. You should start seeing bot visits within 24 hours
๐งช One-Click Test
Test your Cloudflare Worker setup instantly - we'll simulate a GPTBot visit to your domain:
Advanced: Command Line Test
For PowerShell (Windows):
For Terminal (Mac/Linux):
Replace yourdomain.com with your actual domain.
๐ How It Works
Your website works exactly the same. Bots are tracked invisibly.
๐ค AI Bots We Track
โ Frequently Asked Questions
Will this slow down my website?
No! The worker adds less than 1ms of latency. It runs at Cloudflare's edge (closer to your visitors) and tracking is non-blocking (fire-and-forget).
What if Cloudflare or your API goes down?
The worker is designed to fail gracefully. If anything goes wrong, it simply forwards the request to your website normally. Your site will never break because of the tracker.
How much does Cloudflare cost?
The Free plan includes 100,000 requests per day, which is more than enough for bot tracking. You also get free SSL, DDoS protection, and CDN benefits!
Can I use this with WordPress?
Yes! But if you're using WordPress, we recommend our WordPress plugin instead - it's easier to install (just click "Install" in your WordPress admin).
