Dataset Filter & Transform

Returns every filter operator and transform operation this server supports, the order the pipeline runs in, and the maximum rows per call. Call this first if you are unsure what is available. Free, processes no data.

{
  "type": "object",
  "properties": {},
  "additionalProperties": false
}

Cleans a list of JSON rows in one call: transforms each row, filters out the rows you do not want, then optionally sorts, de-duplicates and limits them. Returns the kept rows plus a summary of exactly what was removed and why. Use it to reshape scraped or API data before passing it on: rename and drop fields, cast strings to numbers, compute new fields with arithmetic, extract with regex, then keep only the rows that match your rules. Transforms run before filters, so you can filter on a field you just created.

{
  "type": "object",
  "properties": {
    "rows": {
      "type": "array",
      "description": "The rows to process. Each row is a JSON object. Keys may differ between rows.",
      "items": {
        "type": "object"
      }
    },
    "transforms": {
      "type": "array",
      "description": "Transform steps applied to every row, in order. Each step is an object with an \"op\" key, for example {\"op\":\"cast\",\"field\":\"price\",\"to\":\"number\"} or {\"op\":\"compute\",\"field\":\"total\",\"expression\":\"price * qty\"} or {\"op\":\"rename\",\"from\":\"e_mail\",\"to\":\"email\"}. Call list_capabilities for all 26 operations.",
      "items": {
        "type": "object"
      }
    },
    "filters": {
      "type": "array",
      "description": "Conditions a row must satisfy to be kept, for example {\"field\":\"country\",\"operator\":\"equals\",\"value\":\"UK\"} or {\"field\":\"price\",\"operator\":\"greaterThan\",\"value\":100}. Dot paths like \"address.city\" work. Call list_capabilities for all 25 operators.",
      "items": {
        "type": "object"
      }
    },
    "filterCombineMode": {
      "type": "string",
      "enum": [
        "AND",
        "OR"
      ],
      "description": "Whether a row must match every filter (AND, the default) or any one of them (OR)."
    },
    "caseSensitiveFilters": {
      "type": "boolean",
      "description": "Text comparisons are case-insensitive by default. Set true to make them exact."
    },
    "lenientNumbers": {
      "type": "boolean",
      "description": "On by default: values like \"$1,234.50\", \"49 USD\" and \"12%\" are read as numbers. Set false to require real numeric values."
    },
    "sortBy": {
      "type": "array",
      "description": "Sort the kept rows, for example [{\"field\":\"price\",\"direction\":\"desc\"}].",
      "items": {
        "type": "object"
      }
    },
    "distinctBy": {
      "type": "array",
      "description": "Keep only the first row for each combination of these fields. Applied after sorting, so sorting first lets you keep the newest row per key.",
      "items": {
        "type": "string"
      }
    },
    "offset": {
      "type": "integer",
      "description": "Skip this many rows after filtering and sorting."
    },
    "limit": {
      "type": "integer",
      "description": "Return at most this many rows."
    }
  },
  "required": [
    "rows"
  ],
  "additionalProperties": false
}