HasData Web Scraping
Fetch any public web page through managed proxies, with optional JS rendering and extraction rules.
Sollte ich dies verwenden
Qualität und Sicherheit
Befunde (2)
- LOWin hasdata_web_scraping_web_scraping_scrapeWebPage
- LOWin hasdata_web_scraping_web_scraping_scrapeWebPage
Basierend auf einer automatisierten Analyse der Tool-Definitionen und der Einhaltung des Protokolls.
Kontextkosten
Dies ist die ungefähre Anzahl der Tokens, die jedes Mal verbraucht werden, wenn die Tools des Servers in den Kontext eines Modells geladen werden. Höhere Werte verringern die Aufmerksamkeit, die für andere Aufgaben verfügbar ist.
Installieren
Installation mit einem Klick
Fügen Sie dies Ihrer Datei `claude_desktop_config.json` hinzu:
{
"mcpServers": {
"web-scraping": {
"command": "npx",
"args": [
"@hasdata/web-scraping-mcp"
]
}
}
}Ausführbare Pakete
1.0.2stdio1.0.2stdioRemote-Endpunkte
https://mcp.hasdata.com/api/mcp?apis=web_scrapingstreamable-httphttps://mcp.hasdata.com/mcp?apis=web_scrapingstreamable-httpWas es kann
Tool-Inventar
Tools (1)
🔴hasdata_web_scraping_web_scraping_scrapeWebPage(url, headers, proxyType, proxyCountry, blockResources, ...)
Scrape Web Page Universal web scraper that fetches any public URL through managed proxies (datacenter or residential, geo-targeted) with optional JS rendering, custom headers, wait conditions, jsScenario actions (click, scroll, fill, waitFor), screenshots, resource/ad/URL blocking (these require JS rendering), and extractRules/aiExtractRules for LLM-driven structured extraction. Returns HTML, text, markdown, and/or JSON along with status code, extracted emails and links, CSS-selector extractions, and AI-structured fields per schema. Use as a fallback/universal fetcher for sites without a dedicated API, for scraping JS-heavy SPAs, capturing screenshots, or producing clean markdown/structured JSON to feed downstream parsers, RAG pipelines, or data warehouses.
Eingabe-Schema
{
"type": "object",
"properties": {
"url": {
"type": "string",
"description": "The URL of the web page to scrape."
},
"headers": {
"type": "object",
"additionalProperties": {},
"description": "Optional custom headers to send with the request."
},
"proxyType": {
"type": "string",
"enum": [
"datacenter",
"residential"
],
"description": "Type of proxy to use."
},
"proxyCountry": {
"type": "string",
"enum": [
"US",
"UK",
"DE",
"IE",
"FR",
"IT",
"SE",
"BR",
"CA",
"JP",
"SG",
"IN",
"ID",
"IL"
],
"description": "Optional proxy country code."
},
"blockResources": {
"type": "boolean",
"description": "Whether to block loading of resources like images and stylesheets."
},
"blockAds": {
"type": "boolean",
"description": "Whether to block ads."
},
"blockUrls": {
"type": "array",
"items": {
"type": "string"
},
"description": "List of URLs to block."
},
"wait": {
"type": "number",
"description": "Time in milliseconds to wait after the page load."
},
"waitFor": {
"type": "string",
"description": "CSS selector to wait for before scraping."
},
"jsScenario": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": {}
},
"description": "Enables custom JavaScript interactions on the target webpage during scraping. It's an array where each object defines a specific action or step. These actions can include clicking elements, waiting for elements, executing custom scripts, and more. Key actions within this field include:\n\n- `evaluate`: Run custom JavaScript code on the page.\n- `click`: Click on an element specified by a CSS selector.\n- `wait`: Pause for a set duration (in milliseconds).\n- `waitFor`: Delay until a specific element appears.\n- `waitForAndClick`: Combine waiting for an element and then clicking it.\n- `scrollX`, `scrollY`: Scroll to specified positions on the page.\n- `fill`: Enter values into input fields identified by CSS selectors.\n\nActions are executed sequentially.\n"
},
"extractRules": {
"type": "object",
"additionalProperties": {},
"description": "Rules for extracting specific data from the page. For example: `{ \"title\": \"h1\", \"link_href\": \"a#link @href\", \"page_text\": \"body\" }`\n"
},
"screenshot": {
"type": "boolean",
"description": "Whether to take a screenshot of the page."
},
"jsRendering": {
"type": "boolean",
"description": "Enable JavaScript rendering."
},
"autoMode": {
"type": "boolean",
"description": "Use the cheapest proxy type and JS rendering that load the page, overriding `jsRendering` and `proxyType`. The chosen values are returned in `requestMetadata` and billed."
},
"extractEmails": {
"type": "boolean",
"description": "Extract emails from the page."
},
"extractLinks": {
"type": "boolean",
"description": "Extract links from the page."
},
"includeOnlyTags": {
"type": "array",
"items": {
"type": "string"
},
"description": "The `includeOnlyTags` parameter accepts an array of valid CSS selectors.\nWhen specified, only the elements matching these selectors will be included in the response content.\nEach value must be a valid `querySelectorAll` selector. Useful for extracting specific parts of the document.\n"
},
"excludeTags": {
"type": "array",
"items": {
"type": "string"
},
"description": "The `excludeTags` parameter accepts an array of valid CSS selectors.\nElements matching these selectors will be removed from the final output.\nEach value must be a valid `querySelectorAll` selector. This can be used to remove ads, scripts, or\nother unwanted sections.\n"
},
"removeBase64Images": {
"type": "boolean",
"description": "If set to `true`, any images embedded as base64-encoded strings will be removed from the output.\nUseful for reducing response size or when base64 images are not needed.\n"
},
"outputFormat": {
"type": "array",
"items": {
"type": "string",
"enum": [
"html",
"text",
"markdown",
"json"
]
},
"minItems": 1,
"description": "The outputFormat parameter specifies the desired response format: `html`, `text`, `markdown`, or `json`. If only one of `html`, `text`, or `markdown` is provided, the API returns the response in that format. If multiple formats are specified, the API returns a JSON response with keys for each requested format. If `json` is included with any other format, the API returns a JSON response with keys for the other specified formats.\n"
},
"aiExtractRules": {
"type": "object",
"additionalProperties": {},
"description": "Defines custom rules for AI-based data extraction using LLMs. This enables the system to extract structured data directly from the HTML of the page. Each key in the object represents a desired output field name, and the value specifies its type and optional description to guide the AI.\n\nSupported types:\n- `string`: plain text value\n- `number`: numeric value\n- `boolean`: true/false\n- `list`: an array of values\n- `item`: a nested object with its own structure defined under `output`\n"
}
},
"required": [
"url"
],
"additionalProperties": false,
"$schema": "http://json-schema.org/draft-07/schema#"
}Community
Nachweis