Siftwright web data

Get transcripts (captions) for YouTube videos, Shorts, playlists or channels as plain text and timed segments, with video metadata. Uses the captions YouTube already has; videos without captions return a failed item. Each video whose transcript comes back counts as one result on your Siftwright plan; failures are free.

{
  "type": "object",
  "properties": {
    "urls": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "minItems": 1,
      "maxItems": 10,
      "description": "1 to 10 YouTube video, Shorts, playlist or channel URLs (or 11-character video IDs)."
    },
    "language": {
      "type": "string",
      "description": "Preferred caption language code(s), comma-separated in order of preference. Default en."
    },
    "translateTo": {
      "type": "string",
      "description": "Optional language code to machine-translate the captions into."
    },
    "outputFormats": {
      "type": "array",
      "items": {
        "type": "string",
        "enum": [
          "text",
          "segments",
          "srt",
          "vtt"
        ]
      },
      "description": "Default [\"text\",\"segments\"]."
    },
    "maxVideosPerSource": {
      "type": "integer",
      "minimum": 1,
      "maximum": 25,
      "description": "For playlists and channels. Default 10."
    }
  },
  "required": [
    "urls"
  ]
}

Capture a PNG or JPEG screenshot, or render a PDF, of public web pages in a real browser. Returns a signed fileUrl per page that can be downloaded without a key for 7 days. Each page captured counts as one result.

{
  "type": "object",
  "properties": {
    "urls": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "minItems": 1,
      "maxItems": 5,
      "description": "1 to 5 http(s) URLs."
    },
    "format": {
      "type": "string",
      "enum": [
        "png",
        "jpeg",
        "pdf"
      ],
      "description": "Default png."
    },
    "fullPage": {
      "type": "boolean",
      "description": "Capture the whole page height. Default true."
    },
    "viewportWidth": {
      "type": "integer",
      "minimum": 320,
      "maximum": 3840,
      "description": "Default 1280."
    },
    "viewportHeight": {
      "type": "integer",
      "minimum": 240,
      "maximum": 2160,
      "description": "Default 800."
    },
    "pdfPageFormat": {
      "type": "string",
      "enum": [
        "A4",
        "A3",
        "Letter",
        "Legal"
      ]
    },
    "waitForSelector": {
      "type": "string",
      "description": "CSS selector to wait for before capturing."
    },
    "delaySec": {
      "type": "integer",
      "minimum": 0,
      "maximum": 10
    }
  },
  "required": [
    "urls"
  ]
}

Search Google News for a keyword, company or topic and get articles as structured data (title, link, source, published time). Each article returned counts as one result.

{
  "type": "object",
  "properties": {
    "query": {
      "type": "string",
      "description": "What to search for, up to 300 characters."
    },
    "maxItems": {
      "type": "integer",
      "minimum": 1,
      "maximum": 100,
      "description": "Default 30."
    },
    "language": {
      "type": "string",
      "description": "Default en."
    },
    "country": {
      "type": "string",
      "description": "Two-letter country code. Default US."
    }
  },
  "required": [
    "query"
  ]
}

Check whether domains use post-quantum TLS key exchange (X25519MLKEM768 / ML-KEM), which groups they prefer, TLS 1.2/1.3 support, certificate algorithms and expiry, and HSTS, with a letter grade. Each host that answers counts as one result.

{
  "type": "object",
  "properties": {
    "hosts": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "minItems": 1,
      "maxItems": 20,
      "description": "1 to 20 public domain names (a URL is fine; the hostname is used)."
    }
  },
  "required": [
    "hosts"
  ]
}

Find public emails, phone numbers and social profiles (LinkedIn, X/Twitter, Facebook, Instagram, YouTube) on company websites by crawling up to a few pages of each site. Each website with a successful crawl counts as one result.

{
  "type": "object",
  "properties": {
    "urls": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "minItems": 1,
      "maxItems": 10,
      "description": "1 to 10 company websites or domains."
    },
    "maxPagesPerDomain": {
      "type": "integer",
      "minimum": 1,
      "maximum": 15,
      "description": "Pages to crawl per site. Default 6."
    }
  },
  "required": [
    "urls"
  ]
}

Detect what a website is built with: CMS, ecommerce platform, frameworks, analytics, ad pixels, hosting, CDN, email and DNS provider, with the evidence for each detection. Each website that loads counts as one result.

{
  "type": "object",
  "properties": {
    "urls": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "minItems": 1,
      "maxItems": 20,
      "description": "1 to 20 websites or domains."
    },
    "includeDns": {
      "type": "boolean",
      "description": "Add DNS-based detections (email and DNS provider). Default true."
    }
  },
  "required": [
    "urls"
  ]
}

Get Apple App Store reviews for apps (by name, numeric ID or URL) in one or more countries, filtered by rating, keywords and date. Every 25 reviews returned count as one result (rounded up per request).

{
  "type": "object",
  "properties": {
    "apps": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "minItems": 1,
      "maxItems": 5,
      "description": "1 to 5 app names, App Store IDs or URLs."
    },
    "countries": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "maxItems": 5,
      "description": "Two-letter storefront codes. Default [\"us\"]."
    },
    "maxReviewsPerApp": {
      "type": "integer",
      "minimum": 1,
      "maximum": 2000,
      "description": "Default 200."
    },
    "sortBy": {
      "type": "string",
      "enum": [
        "mostrecent",
        "mosthelpful"
      ]
    },
    "minRating": {
      "type": "integer",
      "minimum": 1,
      "maximum": 5
    },
    "maxRating": {
      "type": "integer",
      "minimum": 1,
      "maximum": 5
    },
    "keywords": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Keep only reviews mentioning any of these words."
    },
    "since": {
      "type": "string",
      "description": "ISO date; keep only reviews on or after it."
    }
  },
  "required": [
    "apps"
  ]
}

List open jobs from company careers pages on Greenhouse, Lever, Ashby, SmartRecruiters and Recruitee in one schema (title, team, location, remote, salary when published, apply URL), with title, location, department and recency filters. Every 3 jobs returned count as one result (rounded up per request).

{
  "type": "object",
  "properties": {
    "companies": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "minItems": 1,
      "maxItems": 10,
      "description": "1 to 10 careers-page URLs, company domains, or ats:slug values like greenhouse:airbnb."
    },
    "titleKeywords": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "excludeTitleKeywords": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "locations": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "departments": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "remoteOnly": {
      "type": "boolean"
    },
    "postedWithinDays": {
      "type": "integer",
      "minimum": 0,
      "maximum": 365,
      "description": "0 = any time."
    },
    "maxJobsPerCompany": {
      "type": "integer",
      "minimum": 1,
      "maximum": 500,
      "description": "Default 100."
    },
    "includeDescription": {
      "type": "boolean",
      "description": "Include full job description text. Default false."
    }
  },
  "required": [
    "companies"
  ]
}

Look up who a domain is registered with, when it was created, when it expires, its status, nameservers and DNSSEC (from RDAP/WHOIS registries), plus A, MX, SPF and DMARC records, and whether it looks available. Every 3 domains with a registry answer count as one result (rounded up per request).

{
  "type": "object",
  "properties": {
    "domains": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "minItems": 1,
      "maxItems": 50,
      "description": "1 to 50 domain names."
    },
    "includeDns": {
      "type": "boolean",
      "description": "Add A, MX, SPF and DMARC records. Default true."
    }
  },
  "required": [
    "domains"
  ]
}