HasData Web Scraping

Fetch any public web page through managed proxies, with optional JS rendering and extraction rules.

사용해야 할까요

품질 및 안전성

B
설명 품질
100%
스키마 완전성
100%
이름 품질
30%
오염 위험
100%
권한 일치
100%
프로토콜 준수
100%

발견 사항 (2)

  • LOWTool 'hasdata_web_scraping_web_scraping_scrapeWebPage' doesn't follow camelCase/snake_casehasdata_web_scraping_web_scraping_scrapeWebPage에서
  • LOWTool 'hasdata_web_scraping_web_scraping_scrapeWebPage' name length outside 3-30 rangehasdata_web_scraping_web_scraping_scrapeWebPage에서

도구 정의와 프로토콜 준수에 대한 자동 분석을 기반으로 합니다.

컨텍스트 비용

~1,242토큰 (도구 정의)
~13.9 KB일반적인 응답 크기
중간 정도의 주의 영향 (128k 컨텍스트의 0.97%)

이는 서버의 도구가 모델의 컨텍스트에 로드될 때마다 소비되는 대략적인 토큰 수입니다. 수치가 높을수록 다른 작업에 사용할 수 있는 주의가 줄어듭니다.

설치

원클릭 설치

`claude_desktop_config.json` 파일에 다음을 추가하세요:

{
  "mcpServers": {
    "web-scraping": {
      "command": "npx",
      "args": [
        "@hasdata/web-scraping-mcp"
      ]
    }
  }
}

실행 가능한 패키지

npm@hasdata/web-scraping-mcp1.0.2stdio
pypihasdata-web-scraping-mcp1.0.2stdio

원격 엔드포인트

https://mcp.hasdata.com/api/mcp?apis=web_scrapingstreamable-http
https://mcp.hasdata.com/mcp?apis=web_scrapingstreamable-http

할 수 있는 일

도구 목록

도구 (1)

🟢 읽기 전용🟡 쓰기🔴 삭제⚪ 알 수 없음
🔴hasdata_web_scraping_web_scraping_scrapeWebPage(url, headers, proxyType, proxyCountry, blockResources, ...)

Scrape Web Page Universal web scraper that fetches any public URL through managed proxies (datacenter or residential, geo-targeted) with optional JS rendering, custom headers, wait conditions, jsScenario actions (click, scroll, fill, waitFor), screenshots, resource/ad/URL blocking (these require JS rendering), and extractRules/aiExtractRules for LLM-driven structured extraction. Returns HTML, text, markdown, and/or JSON along with status code, extracted emails and links, CSS-selector extractions, and AI-structured fields per schema. Use as a fallback/universal fetcher for sites without a dedicated API, for scraping JS-heavy SPAs, capturing screenshots, or producing clean markdown/structured JSON to feed downstream parsers, RAG pipelines, or data warehouses.

입력 스키마

{
  "type": "object",
  "properties": {
    "url": {
      "type": "string",
      "description": "The URL of the web page to scrape."
    },
    "headers": {
      "type": "object",
      "additionalProperties": {},
      "description": "Optional custom headers to send with the request."
    },
    "proxyType": {
      "type": "string",
      "enum": [
        "datacenter",
        "residential"
      ],
      "description": "Type of proxy to use."
    },
    "proxyCountry": {
      "type": "string",
      "enum": [
        "US",
        "UK",
        "DE",
        "IE",
        "FR",
        "IT",
        "SE",
        "BR",
        "CA",
        "JP",
        "SG",
        "IN",
        "ID",
        "IL"
      ],
      "description": "Optional proxy country code."
    },
    "blockResources": {
      "type": "boolean",
      "description": "Whether to block loading of resources like images and stylesheets."
    },
    "blockAds": {
      "type": "boolean",
      "description": "Whether to block ads."
    },
    "blockUrls": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "List of URLs to block."
    },
    "wait": {
      "type": "number",
      "description": "Time in milliseconds to wait after the page load."
    },
    "waitFor": {
      "type": "string",
      "description": "CSS selector to wait for before scraping."
    },
    "jsScenario": {
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": {}
      },
      "description": "Enables custom JavaScript interactions on the target webpage during scraping. It's an array where each object defines a specific action or step. These actions can include clicking elements, waiting for elements, executing custom scripts, and more. Key actions within this field include:\n\n- `evaluate`: Run custom JavaScript code on the page.\n- `click`: Click on an element specified by a CSS selector.\n- `wait`: Pause for a set duration (in milliseconds).\n- `waitFor`: Delay until a specific element appears.\n- `waitForAndClick`: Combine waiting for an element and then clicking it.\n- `scrollX`, `scrollY`: Scroll to specified positions on the page.\n- `fill`: Enter values into input fields identified by CSS selectors.\n\nActions are executed sequentially.\n"
    },
    "extractRules": {
      "type": "object",
      "additionalProperties": {},
      "description": "Rules for extracting specific data from the page. For example: `{ \"title\": \"h1\", \"link_href\": \"a#link @href\", \"page_text\": \"body\" }`\n"
    },
    "screenshot": {
      "type": "boolean",
      "description": "Whether to take a screenshot of the page."
    },
    "jsRendering": {
      "type": "boolean",
      "description": "Enable JavaScript rendering."
    },
    "autoMode": {
      "type": "boolean",
      "description": "Use the cheapest proxy type and JS rendering that load the page, overriding `jsRendering` and `proxyType`. The chosen values are returned in `requestMetadata` and billed."
    },
    "extractEmails": {
      "type": "boolean",
      "description": "Extract emails from the page."
    },
    "extractLinks": {
      "type": "boolean",
      "description": "Extract links from the page."
    },
    "includeOnlyTags": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "The `includeOnlyTags` parameter accepts an array of valid CSS selectors.\nWhen specified, only the elements matching these selectors will be included in the response content.\nEach value must be a valid `querySelectorAll` selector. Useful for extracting specific parts of the document.\n"
    },
    "excludeTags": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "The `excludeTags` parameter accepts an array of valid CSS selectors.\nElements matching these selectors will be removed from the final output.\nEach value must be a valid `querySelectorAll` selector. This can be used to remove ads, scripts, or\nother unwanted sections.\n"
    },
    "removeBase64Images": {
      "type": "boolean",
      "description": "If set to `true`, any images embedded as base64-encoded strings will be removed from the output.\nUseful for reducing response size or when base64 images are not needed.\n"
    },
    "outputFormat": {
      "type": "array",
      "items": {
        "type": "string",
        "enum": [
          "html",
          "text",
          "markdown",
          "json"
        ]
      },
      "minItems": 1,
      "description": "The outputFormat parameter specifies the desired response format: `html`, `text`, `markdown`, or `json`. If only one of `html`, `text`, or `markdown` is provided, the API returns the response in that format. If multiple formats are specified, the API returns a JSON response with keys for each requested format. If `json` is included with any other format, the API returns a JSON response with keys for the other specified formats.\n"
    },
    "aiExtractRules": {
      "type": "object",
      "additionalProperties": {},
      "description": "Defines custom rules for AI-based data extraction using LLMs. This enables the system to extract structured data directly from the HTML of the page. Each key in the object represents a desired output field name, and the value specifies its type and optional description to guide the AI.\n\nSupported types:\n- `string`: plain text value\n- `number`: numeric value\n- `boolean`: true/false\n- `list`: an array of values\n- `item`: a nested object with its own structure defined under `output`\n"
    }
  },
  "required": [
    "url"
  ],
  "additionalProperties": false,
  "$schema": "http://json-schema.org/draft-07/schema#"
}

커뮤니티

이 서버 평가하기

증거

최근 관측

검증됨버전이 기록되지 않음도구 1개
검증됨버전이 기록되지 않음도구 1개
검증됨버전이 기록되지 않음도구 1개
검증됨버전이 기록되지 않음도구 1개
검증됨버전이 기록되지 않음도구 1개
검증됨버전이 기록되지 않음도구 1개
검증됨버전이 기록되지 않음도구 1개
검증됨버전이 기록되지 않음도구 1개
인증 필요버전이 기록되지 않음—
인증 필요버전이 기록되지 않음—