AI Robots.txt Generator

Control which AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access your site. Generate a production-ready robots.txt in seconds.

Preset

AI Crawlers

GPTBot (OpenAI)ChatGPT-UserClaudeBot (Anthropic)Claude-WebGoogle-Extended (Gemini)PerplexityBotCCBot (Common Crawl)Bytespider (ByteDance)Applebot-ExtendedAmazonbotcohere-ai

Search Engines

GooglebotBingbotDuckDuckBotYandexBotBaiduspider

SEO Bots

AhrefsBotSemrushBot

Sitemap URL (optional)

Custom Rules (optional)

Preview

# robots.txt generated by GEOKit
# 2026-07-15

User-agent: *
Allow: /

How to deploy:

Download as robots.txt
Upload to your website root directory
Verify at yoursite.com/robots.txt
Test with Google Search Console