Robots.txt Generator
Build a robots.txt with crawler rules, sitemaps, and presets like blocking AI bots.
Download
Presets:
Allow all
Block all
WordPress
Block AI bots
User-agent
Crawl-delay
Allow (one path per line)
Disallow (one path per line)
Add user-agent rule
Sitemap URLs (one per line)
robots.txt
Copy
User-agent: * Disallow: