robotstxt-ai

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

Install

Add this to your `claude_desktop_config.json` file:

{
  "mcpServers": {
    "robotstxt-ai": {
      "command": "npx",
      "args": [
        "robotstxt-ai-mcp"
      ]
    }
  }
}

Runnable packages

npmrobotstxt-ai-mcp1.0.1stdio

What it can do

No publishable tool enumeration has been recorded.

Community

Evidence

This server has Registry metadata but no public verification run yet.