Rat Radar

Can I scrape this site, and how? Returns a verdict (open, guarded, hostile, blocked), a hostility score, a recommended approach with steps and risks, bot protections, robots.txt and AI-crawler policy, machine-readable surfaces (sitemap, llms.txt, JSON-LD), and eu_ai_training: the signals the EDPB draft Guidelines 03/2026 on web scraping for generative AI count against collecting personal data (robots.txt, ai.txt, CAPTCHAs, login walls), as none, some or strong. Signals, not legal advice. Uses a weekly-probed dataset of several hundred sites; for any other domain it probes live from a datacenter IP.

{
  "type": "object",
  "properties": {
    "domain": {
      "type": "string",
      "description": "A domain such as reddit.com"
    }
  },
  "required": [
    "domain"
  ]
}

What should I build with this site, and how? Give a domain and, optionally, a goal in plain words (for example "price alerts", "sales leads", "new jobs", "change log", "knowledge base"). Returns the best-matching workflow for that site: what it does, its cadence, a trigger, collect, shape, store, act pipeline, the fields to keep, and step-by-step build recipes for n8n, Make, Zapier and Apify (or just one with platform). Workflows marked found_on_this_site come from what the weekly probe saw on the site itself. Also returns the other workflows, the golden data, and the guardrails.

{
  "type": "object",
  "properties": {
    "domain": {
      "type": "string",
      "description": "A domain such as amazon.com"
    },
    "goal": {
      "type": "string",
      "description": "optional, what you want to achieve"
    },
    "platform": {
      "type": "string",
      "enum": [
        "n8n",
        "make",
        "zapier",
        "apify"
      ],
      "description": "optional, only return this platform (n8n also returns n8n_workflow, an importable file):'s recipe"
    }
  },
  "required": [
    "domain"
  ]
}

List the sites in the dataset, optionally by category or verdict. Use it to find easy targets or to see what is covered.

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "description": "e.g. shop, jobs, news, property, travel, gov"
    },
    "verdict": {
      "type": "string",
      "enum": [
        "open",
        "guarded",
        "hostile",
        "blocked"
      ]
    },
    "limit": {
      "type": "integer",
      "description": "default 50, max 200"
    }
  }
}

How do I extract the data from THIS page? Fetches one public page and returns a ranked extraction plan: embedded JSON, JSON-LD, repeated records with CSS selectors and field names, tables, feeds, hidden API calls. Each source has a reliability, proof rows pulled from the real page, and the plan notes pagination and blockers. Set include_code for a starter Python script. Works on the raw HTML only, so JavaScript-built pages report a blocker instead of data. Use check_site first to learn whether the site allows scraping at all.

{
  "type": "object",
  "properties": {
    "url": {
      "type": "string",
      "description": "Full page URL, e.g. https://example.com/products"
    },
    "include_code": {
      "type": "boolean",
      "description": "also return a starter Python script"
    }
  },
  "required": [
    "url"
  ]
}

Draw a page's anatomy as an Excalidraw diagram: its regions, the ranked ways to get its data, proof rows, the best route and what the page is made of. Returns a download link for the .excalidraw file (give it to the user; it opens at excalidraw.com or in Obsidian) and a file_name. With include_scene, also the scene itself (a few hundred KB).

{
  "type": "object",
  "properties": {
    "url": {
      "type": "string",
      "description": "Full page URL, e.g. https://example.com/products"
    },
    "include_scene": {
      "type": "boolean",
      "description": "also return the full Excalidraw scene JSON (large). Default false: just the download link"
    }
  },
  "required": [
    "url"
  ]
}

Find sites in the dataset by a word: a name, host, category (jobs, shop, news, gov), country or verdict. Use it to discover what is covered before calling check_site.

{
  "type": "object",
  "properties": {
    "query": {
      "type": "string"
    },
    "limit": {
      "type": "integer",
      "description": "default 25, max 100"
    }
  },
  "required": [
    "query"
  ]
}

Does robots.txt allow fetching this exact URL? Fetches the live robots.txt and applies the longest-match rule for a user agent (default: * plus GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot). Returns allowed or not, the rule that decided it, and any crawl delay. A courtesy check, not a legal opinion.

{
  "type": "object",
  "properties": {
    "url": {
      "type": "string",
      "description": "The full URL you want to fetch"
    },
    "user_agent": {
      "type": "string",
      "description": "optional, e.g. GPTBot"
    }
  },
  "required": [
    "url"
  ]
}

Headline numbers across the dataset: verdict mix, how many sites block AI crawlers (and which), protection vendors, by category and country.

{
  "type": "object",
  "properties": {}
}