Robots.txt Generator — Free
Generate a robots.txt file for your website. Control which bots can crawl which pages.
Quick Presets
Rules
Rule 1
User-agent: * Allow: / Disallow: /admin/
🤖
Block AI crawlers
One-click preset to block GPTBot, ClaudeBot, CCBot and other AI scrapers.
⚡
Live preview
Output updates instantly as you change rules or options.
📥
Download ready
Download the file and place it at the root of your website.