Arquestra Audit

Proposes how to read a data file for an audit, from its column names (optionally a type and one example value each) and the results the person wants to publish. Returns a proposed spec (the role of each column, each result written as a claim), how claims are written, the catalog of checks and the points the person has to confirm. It receives no rows.

{
  "type": "object",
  "properties": {
    "columns": {
      "type": "array",
      "maxItems": 200,
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string"
          },
          "type": {
            "type": "string"
          },
          "example": {
            "type": "string"
          }
        },
        "required": [
          "name"
        ],
        "additionalProperties": false
      }
    },
    "dataset_name": {
      "type": "string"
    },
    "figures": {
      "type": "array",
      "maxItems": 40,
      "items": {
        "type": "string"
      },
      "description": "The results the person wants to publish, in their words, e.g. 'share of projects per sector'."
    }
  },
  "required": [
    "columns"
  ],
  "additionalProperties": false
}

Opens the audit panel in the chat. Without a spec, the panel receives the person's file, reads it in their browser and returns a profile of it: per column its kind, empty cells, distinct values, the values of categories with their counts, the range of numbers and dates, and the first rows. With a confirmed spec, the panel measures that same file in the browser with measure.py and shows one light per result. The file is not sent to this server.

{
  "type": "object",
  "properties": {
    "spec": {
      "type": "object",
      "description": "The spec from start_audit, as confirmed by the person. Leave it out to open the panel first, to receive the file."
    },
    "columns": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Optional: the column names of the file, to check the spec refers to real columns."
    }
  },
  "additionalProperties": false
}

Returns measure.py with a confirmed spec written in: a fixed Python script (standard library, no network, fixed seeds) that counts what each check needs from a data file and prints those counts with its own fingerprint and no rows. Reads .csv or .json. If the spec has errors, returns the errors instead.

{
  "type": "object",
  "properties": {
    "spec": {
      "type": "object",
      "description": "The spec from start_audit, as confirmed by the person."
    },
    "columns": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Optional: the column names of the file, to check the spec refers to real columns."
    }
  },
  "required": [
    "spec"
  ],
  "additionalProperties": false
}

Turns the output of measure.py into one light per result (red, amber or green, with its reasons), after checking that the output comes from the unchanged script and from the given spec. Shows the lights in the chat and returns them, with a link to the same view on arquestra.dev that carries only codes. Stores nothing.

{
  "type": "object",
  "properties": {
    "spec": {
      "type": "object",
      "description": "The same spec sent to prepare_script."
    },
    "output": {
      "type": "string",
      "description": "Everything measure.py printed: the fingerprint line and the measures line."
    }
  },
  "required": [
    "spec",
    "output"
  ],
  "additionalProperties": false
}

Published example audits, such as the AI Project Radar, and how the audit works. With no arguments: the overview and the list of examples. With a name: one example, result by result (level filters by light). With a name and an id: one result with every reason and what to change. They are not the person's data.

{
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "description": "Example name from the list, e.g. 'radar'."
    },
    "id": {
      "type": "string",
      "description": "Figure id inside that example."
    },
    "level": {
      "type": "string",
      "enum": [
        "red",
        "amber",
        "green"
      ],
      "description": "Only figures with this light."
    },
    "lang": {
      "type": "string",
      "enum": [
        "en",
        "es"
      ],
      "description": "Language of the texts. Default en."
    }
  },
  "additionalProperties": false
}