LLMS Central - The Robots.txt for AI
tutorials9 min read

How to Add llms.txt to WordPress: Complete Step-by-Step Guide

By LLMS Central Team

How to Add llms.txt to WordPress: Complete Step-by-Step Guide

WordPress powers 43% of all websites, making it the most important platform for llms.txt implementation. This guide shows you exactly how to add and optimize llms.txt on your WordPress site.

Why WordPress Sites Need llms.txt

Growing Importance

AI Search Traffic:

  • 25% of searches now AI-powered
  • ChatGPT has 200M+ weekly users
  • Perplexity growing 50%+ annually
  • Early adopters gaining advantage

WordPress Advantages:

  • Large content libraries
  • SEO-friendly structure
  • Easy plugin integration
  • Built-in XML sitemaps

Three Methods to Add llms.txt

Method 1: Plugin Installation (Easiest)

Best for: Non-technical users, quick setup

Step 1: Install Plugin

\\\`

1. Go to WordPress Admin → Plugins → Add New

2. Search for "LLMS Central Bot Tracker"

3. Click "Install Now"

4. Click "Activate"

\\\`

Step 2: Configure Settings

\\\`

1. Go to Settings → LLMS Central

2. Enter your site description

3. Select priority content

4. Click "Save Changes"

\\\`

Step 3: Verify Installation

Visit: \https://yoursite.com/llms.txt\

Plugin Features:

✅ Automatic llms.txt generation

✅ Bot tracking and analytics

✅ Auto-updates when content changes

✅ Dashboard widget

✅ No coding required

Method 2: Manual File Upload

Best for: Users who want full control

Step 1: Create llms.txt File

Create a text file named \llms.txt\:

\\\`

Your Blog Name

> Expert insights on [your niche]

Featured Content

  • [Best Guide](https://yoursite.com/best-guide): Comprehensive tutorial
  • [Popular Post](https://yoursite.com/popular): Most-read article
  • [Case Study](https://yoursite.com/case-study): Real results

Blog Categories

  • [Category 1](https://yoursite.com/category/cat1): Topic description
  • [Category 2](https://yoursite.com/category/cat2): Topic description

About

  • [About Us](https://yoursite.com/about): Our story
  • [Contact](https://yoursite.com/contact): Get in touch

\\\`

Step 2: Upload via FTP

\\\`

1. Connect to your site via FTP (FileZilla, etc.)

2. Navigate to public_html or www folder

3. Upload llms.txt to root directory

4. Set permissions to 644

\\\`

Step 3: Verify

Visit: \https://yoursite.com/llms.txt\

Method 3: Code Snippet (Advanced)

Best for: Developers, dynamic content

Step 1: Install Code Snippets Plugin

\\\`

1. Install "Code Snippets" plugin

2. Activate plugin

3. Go to Snippets → Add New

\\\`

Step 2: Add Custom Function

\\\`php

// Generate llms.txt dynamically

add_action('init', 'serve_llms_txt');

function serve_llms_txt() {

if ($_SERVER['REQUEST_URI'] === '/llms.txt') {

header('Content-Type: text/plain; charset=utf-8');

$site_name = get_bloginfo('name');

$site_desc = get_bloginfo('description');

$site_url = home_url();

echo "# $site_name\n";

echo "> $site_desc\n\n";

// Get recent posts

echo "## Recent Posts\n";

$recent_posts = get_posts([

'numberposts' => 10,

'post_status' => 'publish'

]);

foreach ($recent_posts as $post) {

$title = $post->post_title;

$url = get_permalink($post->ID);

$excerpt = wp_trim_words($post->post_excerpt ?: $post->post_content, 15);

echo "- $title: $excerpt\n";

}

// Get categories

echo "\n## Categories\n";

$categories = get_categories(['hide_empty' => true]);

foreach ($categories as $cat) {

$name = $cat->name;

$url = get_category_link($cat->term_id);

$desc = $cat->description ?: "Articles about $name";

echo "- $name: $desc\n";

}

// About section

echo "\n## About\n";

echo "- About Us: Learn more about us\n";

echo "- Contact: Get in touch\n";

exit;

}

}

?>

\\\`

Step 3: Save and Activate

\\\`

1. Title: "Dynamic llms.txt Generator"

2. Click "Save Changes and Activate"

3. Test at yoursite.com/llms.txt

\\\`

Optimizing Your WordPress llms.txt

1. Content Selection

Priority Content to Include:

High-Value Posts:

\\\`php

// Get most popular posts

$popular_posts = new WP_Query([

'posts_per_page' => 10,

'meta_key' => 'post_views_count',

'orderby' => 'meta_value_num',

'order' => 'DESC'

]);

\\\`

Evergreen Content:

  • Comprehensive guides
  • Tutorials and how-tos
  • Resource pages
  • Best practices

Recent Content:

  • Latest blog posts
  • Updated articles
  • New resources
  • Current trends

2. Category Organization

Logical Structure:

\\\`

Main Topics

  • [SEO](https://yoursite.com/category/seo): Search optimization guides
  • [Content Marketing](https://yoursite.com/category/content): Content strategies
  • [Analytics](https://yoursite.com/category/analytics): Data and insights

Resources

  • [Tools](https://yoursite.com/tools): Free tools and calculators
  • [Templates](https://yoursite.com/templates): Downloadable templates
  • [Guides](https://yoursite.com/guides): Step-by-step guides

\\\`

3. Automatic Updates

Keep llms.txt Fresh:

\\\`php

// Update llms.txt when posts are published

add_action('publish_post', 'update_llms_txt_cache');

add_action('save_post', 'update_llms_txt_cache');

function update_llms_txt_cache($post_id) {

// Clear any caching

if (function_exists('wp_cache_flush')) {

wp_cache_flush();

}

// Regenerate llms.txt

// Your generation logic here

}

\\\`

WordPress-Specific Optimizations

1. Permalink Structure

SEO-Friendly URLs:

\\\`

Settings → Permalinks → Post name

✅ https://yoursite.com/post-title

❌ https://yoursite.com/?p=123

\\\`

2. XML Sitemap Integration

Reference Your Sitemap:

\\\`

Your Blog

> Description

Sitemap

  • [XML Sitemap](https://yoursite.com/sitemap.xml): Complete site structure
  • [Post Sitemap](https://yoursite.com/post-sitemap.xml): All blog posts

\\\`

3. robots.txt Configuration

Allow AI Bots:

\\\`

WordPress robots.txt

User-agent: *

Allow: /wp-content/uploads/

Disallow: /wp-admin/

Disallow: /wp-includes/

AI Bots

User-agent: GPTBot

Allow: /

Crawl-delay: 1

User-agent: ClaudeBot

Allow: /

Crawl-delay: 1

User-agent: PerplexityBot

Allow: /

Crawl-delay: 1

Sitemap: https://yoursite.com/sitemap.xml

\\\`

Edit in WordPress:

  • Use Yoast SEO or Rank Math
  • Or edit via FTP at root directory

4. Caching Considerations

Exclude llms.txt from Cache:

WP Rocket:

\\\`

Settings → Advanced → Never cache URL(s)

Add: /llms.txt

\\\`

W3 Total Cache:

\\\`

Performance → Page Cache → Never cache the following pages

Add: llms.txt

\\\`

Cloudflare:

\\\`

Page Rules → Create Page Rule

URL: yoursite.com/llms.txt

Setting: Cache Level = Bypass

\\\`

Tracking AI Bot Visits

Install Bot Tracking

Option 1: Plugin Method

\\\`

1. LLMS Central plugin includes bot tracking

2. View analytics in WordPress admin

3. Dashboard widget shows recent visits

\\\`

Option 2: JavaScript Widget

Add to theme's footer.php:

\\\`php

// Add before tag

function add_bot_tracker() {

?>

}

add_action('wp_footer', 'add_bot_tracker');

?>

\\\`

Analytics Dashboard

View in WordPress:

  • Dashboard → LLMS Central
  • See bot visits by type
  • Track most crawled pages
  • Monitor trends over time

Common WordPress Issues

Issue 1: 404 Error on llms.txt

Causes:

  • Permalink structure issues
  • .htaccess problems
  • Plugin conflicts

Solutions:

\\\`

1. Go to Settings → Permalinks

2. Click "Save Changes" (refresh permalinks)

3. Clear all caches

4. Test again

\\\`

Manual .htaccess Fix:

\\\`apache

Add to .htaccess

ForceType text/plain

\\\`

Issue 2: Wrong Content Type

Problem: llms.txt shows as HTML

Solution:

\\\`php

// Force text/plain

add_action('template_redirect', 'force_llms_txt_content_type');

function force_llms_txt_content_type() {

if (is_404() && strpos($_SERVER['REQUEST_URI'], 'llms.txt') !== false) {

header('Content-Type: text/plain; charset=utf-8');

readfile(ABSPATH . 'llms.txt');

exit;

}

}

\\\`

Issue 3: Plugin Conflicts

Common Conflicts:

  • Security plugins blocking bots
  • Cache plugins serving old content
  • SEO plugins interfering

Troubleshooting:

\\\`

1. Deactivate all plugins except LLMS Central

2. Test llms.txt

3. Reactivate plugins one by one

4. Identify conflicting plugin

5. Configure or replace

\\\`

Multisite WordPress

Network-Wide Implementation

For WordPress Multisite:

\\\`php

// Add to mu-plugins folder

add_action('init', 'multisite_llms_txt');

function multisite_llms_txt() {

if ($_SERVER['REQUEST_URI'] === '/llms.txt') {

header('Content-Type: text/plain; charset=utf-8');

$site_id = get_current_blog_id();

$site_details = get_blog_details($site_id);

echo "# {$site_details->blogname}\n";

echo "> {$site_details->blogdescription}\n\n";

// Site-specific content

switch_to_blog($site_id);

// Your content generation logic

restore_current_blog();

exit;

}

}

\\\`

Best Practices for WordPress

1. Content Organization

Logical Structure:

\\\`

Pillar Content

  • [Ultimate Guide](URL): Comprehensive resource
  • [Complete Tutorial](URL): Step-by-step guide

Popular Posts

  • [Post 1](URL): Description
  • [Post 2](URL): Description

Categories

  • [Category 1](URL): Topic overview
  • [Category 2](URL): Topic overview

Resources

  • [Tools](URL): Free tools
  • [Downloads](URL): Templates and guides

\\\`

2. Regular Updates

Update Schedule:

  • Weekly: Add new posts
  • Monthly: Review and update descriptions
  • Quarterly: Reorganize structure
  • Annually: Complete audit

3. Performance

Optimize for Speed:

  • Use lightweight plugins
  • Minimize external requests
  • Enable caching (except llms.txt)
  • Optimize images
  • Use CDN

4. Security

Protect Your Site:

  • Keep WordPress updated
  • Use security plugins
  • Monitor bot activity
  • Block malicious bots
  • Regular backups

Advanced WordPress Integration

Custom Post Types

Include Custom Content:

\\\`php

// Add custom post types to llms.txt

$args = [

'post_type' => ['post', 'portfolio', 'case-study'],

'posts_per_page' => 20,

'post_status' => 'publish'

];

$query = new WP_Query($args);

\\\`

WooCommerce Integration

For E-commerce Sites:

\\\`

Products

  • [Product Category 1](URL): Product description
  • [Product Category 2](URL): Product description

Guides

  • [Buying Guide](URL): How to choose
  • [Setup Guide](URL): Installation help

\\\`

Multilingual Sites

WPML/Polylang Support:

\\\`php

// Generate language-specific llms.txt

$current_lang = apply_filters('wpml_current_language', NULL);

if ($current_lang === 'es') {

echo "# Tu Sitio\n";

echo "> Descripción en español\n\n";

} else {

echo "# Your Site\n";

echo "> English description\n\n";

}

\\\`

Testing and Validation

Validation Checklist

File Accessibility

  • [ ] Visit yoursite.com/llms.txt
  • [ ] Loads as plain text
  • [ ] No 404 errors
  • [ ] Correct content type

Content Quality

  • [ ] Clear site description
  • [ ] Working URLs
  • [ ] Logical organization
  • [ ] Recent content included

Technical Setup

  • [ ] robots.txt allows bots
  • [ ] No cache issues
  • [ ] Fast loading
  • [ ] Mobile-friendly

Testing Tools

Use LLMS Central Validator:

\\\`

1. Go to llmscentral.com/submit

2. Enter your domain

3. Get validation report

4. Fix any issues

\\\`

Maintenance Schedule

Weekly Tasks

  • [ ] Check bot visit logs
  • [ ] Verify llms.txt loads
  • [ ] Monitor analytics

Monthly Tasks

  • [ ] Update featured content
  • [ ] Add new posts
  • [ ] Review descriptions
  • [ ] Check for broken links

Quarterly Tasks

  • [ ] Reorganize structure
  • [ ] Update categories
  • [ ] Audit all URLs
  • [ ] Optimize performance

Conclusion

Adding llms.txt to WordPress is straightforward with the right approach. Whether you use a plugin, manual upload, or custom code, the key is to create a well-structured file that accurately represents your best content.

Start with the plugin method for the easiest setup, then customize as needed. Monitor bot activity to see which content AI systems prefer, and continuously optimize based on the data.

---

*Ready to add llms.txt to your WordPress site? Download our free WordPress plugin or use our online generator.*

📚Related Articles