rubrkit
MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
Sollte ich dies verwenden
Qualität und Sicherheit
Befunde (5)
- LOWTool 'rubrkit_hard_delete_artifact_bundle' name length outside 3-30 rangein rubrkit_hard_delete_artifact_bundle
- LOWTool 'rubrkit_list_artifact_bundle_versions' name length outside 3-30 rangein rubrkit_list_artifact_bundle_versions
- LOWTool 'rubrkit_restore_artifact_bundle_version' name length outside 3-30 rangein rubrkit_restore_artifact_bundle_version
- LOWTool 'rubrkit_read_drift_observations' name length outside 3-30 rangein rubrkit_read_drift_observations
- LOWTool 'rubrkit_set_drift_monitor_status' name length outside 3-30 rangein rubrkit_set_drift_monitor_status
Basierend auf einer automatisierten Analyse der Tool-Definitionen und der Einhaltung des Protokolls.
Kontextkosten
Dies ist die ungefähre Anzahl der Tokens, die jedes Mal verbraucht werden, wenn die Tools des Servers in den Kontext eines Modells geladen werden. Höhere Werte verringern die Aufmerksamkeit, die für andere Aufgaben verfügbar ist.
Installieren
Installation mit einem Klick
Fügen Sie dies Ihrer Datei `claude_desktop_config.json` hinzu:
{
"mcpServers": {
"rubrkit": {
"url": "https://www.rubrkit.com/api/v1/mcp"
}
}
}Remote-Endpunkte
https://www.rubrkit.com/api/v1/mcpstreamable-httpWas es kann
Tool-Inventar
Tools (39)
🟢rubrkit_search_api_docs(query, limit)
Search the canonical docs/api Markdown guides and OpenAPI operation summaries.
Eingabe-Schema
{
"type": "object",
"properties": {
"query": {
"type": "string",
"minLength": 1
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 20,
"default": 8
}
},
"required": [
"query"
]
}🟢rubrkit_read_api_doc(slugOrPath)
Read a docs/api Markdown guide or OpenAPI source by slug, file name, or docs/api path.
Eingabe-Schema
{
"type": "object",
"properties": {
"slugOrPath": {
"type": "string",
"minLength": 1
}
},
"required": [
"slugOrPath"
]
}🟢rubrkit_read_openapi_contract
Read the canonical docs/api/openapi.yaml contract.
Eingabe-Schema
{
"type": "object",
"properties": {}
}🟡rubrkit_create_artifact_bundle(name, description, customRubric, settings)
Create an artifact bundle through /api/v1/artifact-bundles. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"name": {
"type": "string",
"minLength": 1,
"maxLength": 120
},
"description": {
"type": [
"string",
"null"
],
"maxLength": 2000
},
"customRubric": {
"type": "object",
"additionalProperties": true
},
"settings": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"name"
]
}🟢rubrkit_list_artifact_bundles(status, limit)
List artifact bundles through /api/v1/artifact-bundles. Requires artifact_bundles:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"active",
"archived",
"all"
],
"default": "active"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 100
}
}
}🟢rubrkit_read_artifact_bundle(artifactBundleId)
Read artifact bundle details, files, and file tree. Requires artifact_bundles:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🔴rubrkit_delete_artifact_bundle(artifactBundleId)
Archive an artifact bundle (soft delete) through DELETE /api/v1/artifact-bundles/{artifactBundleId}. The bundle moves to the archived state and can be restored or permanently deleted later. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🔴rubrkit_hard_delete_artifact_bundle(artifactBundleId)
Permanently delete an archived artifact bundle and all of its files, versions, and run history through DELETE /api/v1/artifact-bundles/{artifactBundleId}/permanent. This is irreversible and only works on bundles that are already archived. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🟡rubrkit_upload_artifact_files(artifactBundleId, files)
Upload one or more text files into an artifact bundle. Requires files:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"files": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"required": [
"path",
"content"
],
"properties": {
"path": {
"type": "string",
"minLength": 1
},
"content": {
"type": "string"
},
"mediaType": {
"type": "string"
},
"artifactType": {
"type": "string"
},
"message": {
"type": "string",
"maxLength": 500
}
}
}
}
},
"required": [
"artifactBundleId",
"files"
]
}🟢rubrkit_list_artifact_files(artifactBundleId, limit)
List files in an artifact bundle. Requires files:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 200,
"default": 200
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_artifact_file(artifactBundleId, fileId)
Read the latest file metadata and content. Requires files:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"fileId"
]
}🟡rubrkit_update_artifact_file(artifactBundleId, fileId, content, mediaType, message)
Write a new editable version for a file. Requires files:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
},
"content": {
"type": "string"
},
"mediaType": {
"type": "string"
},
"message": {
"type": "string",
"maxLength": 500
}
},
"required": [
"artifactBundleId",
"fileId",
"content"
]
}⚪rubrkit_compress_artifact_file(artifactBundleId, fileId, level, dryRun)
Deterministically shorten a markdown or text prompt without losing instructions (free, no credits). Saves a new version unless dryRun is true. Requires files:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
},
"level": {
"type": "string",
"enum": [
"safe",
"aggressive"
],
"default": "safe"
},
"dryRun": {
"type": "boolean",
"default": false
}
},
"required": [
"artifactBundleId",
"fileId"
]
}⚪rubrkit_start_audit(artifactBundleId, targetFileId, targetVersionNumber, artifactBundleVersionNumber, rubricKey, ...)
Start an async audit job. Audit results are cached per account by a content hash of their inputs; pass force: true to bypass the cache and run a fresh audit (the same as the CLI's --no-cache flag). Requires audits:run for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"targetFileId": {
"type": "string",
"format": "uuid"
},
"targetVersionNumber": {
"type": "integer",
"minimum": 1
},
"artifactBundleVersionNumber": {
"type": "integer",
"minimum": 1
},
"rubricKey": {
"type": "string"
},
"artifactType": {
"type": "string"
},
"customRequirements": {
"type": "array",
"items": {
"type": "string"
}
},
"modelTier": {
"type": "string",
"enum": [
"cheap",
"standard",
"premium"
]
},
"reasoningEffort": {
"type": "string",
"enum": [
"minimal",
"low",
"medium",
"high"
]
},
"force": {
"type": "boolean",
"default": false
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_list_audits(artifactBundleId, limit)
List persisted audit history. Requires artifact_bundles:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 25
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_audit(artifactBundleId, auditRunId)
Read a single audit run. Requires artifact_bundles:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"auditRunId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"auditRunId"
]
}⚪rubrkit_apply_audit_rewrite(artifactBundleId, auditRunId)
Apply a completed audit's AI rewrite to the bundle files (a new version per changed file). Requires audits:run for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"auditRunId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"auditRunId"
]
}🟢rubrkit_poll_job(jobId)
Read async job progress. Requires jobs:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"jobId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"jobId"
]
}🟢rubrkit_list_job_events(jobId, limit)
Read append-only job progress events. Requires jobs:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"jobId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 100
}
},
"required": [
"jobId"
]
}🟢rubrkit_list_artifact_bundle_versions(artifactBundleId, limit)
List bundle-level version events. Requires artifact_bundles:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 100
}
},
"required": [
"artifactBundleId"
]
}⚪rubrkit_restore_artifact_bundle_version(artifactBundleId, versionNumber, message)
Restore files from a prior artifact-bundle version by creating new file versions. Requires artifact_bundles:write and files:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"versionNumber": {
"type": "integer",
"minimum": 1
},
"message": {
"type": "string",
"maxLength": 500
}
},
"required": [
"artifactBundleId",
"versionNumber"
]
}🟢rubrkit_list_file_versions(artifactBundleId, fileId, limit)
List immutable versions for a file. Requires files:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 100
}
},
"required": [
"artifactBundleId",
"fileId"
]
}⚪rubrkit_restore_file_version(artifactBundleId, fileId, versionNumber, message)
Restore a prior file version by creating a new version. Requires files:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
},
"versionNumber": {
"type": "integer",
"minimum": 1
},
"message": {
"type": "string",
"maxLength": 500
}
},
"required": [
"artifactBundleId",
"fileId",
"versionNumber"
]
}🟡rubrkit_run_evals(artifactBundleId, auditRunId, sourceFileVersionId, sourceFileId, sourceVersionNumber, ...)
Start an async eval run. Requires evals:run for API keys. Set mode: "advanced" with baselineVersionId and candidateVersionId to run a statistically-backed A/B (repeats defaults to 3, max 5).
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"auditRunId": {
"type": "string",
"format": "uuid"
},
"sourceFileVersionId": {
"type": "string",
"format": "uuid"
},
"sourceFileId": {
"type": "string",
"format": "uuid"
},
"sourceVersionNumber": {
"type": "integer",
"minimum": 1
},
"candidateFileVersionId": {
"type": "string",
"format": "uuid"
},
"candidateFileId": {
"type": "string",
"format": "uuid"
},
"candidateVersionNumber": {
"type": "integer",
"minimum": 1
},
"rubricVersion": {
"type": "string"
},
"maxCases": {
"type": "integer",
"minimum": 1,
"maximum": 20
},
"testCases": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"mode": {
"type": "string",
"enum": [
"standard",
"advanced"
]
},
"repeats": {
"type": "integer",
"minimum": 1,
"maximum": 5
},
"baselineVersionId": {
"type": "string",
"format": "uuid"
},
"candidateVersionId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_list_evals(artifactBundleId, limit)
List eval history. Requires artifact_bundles:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 25
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_eval(artifactBundleId, evalRunId)
Read one eval run, including the `stats` statistical verdict on advanced-mode runs (empty for standard runs). Requires artifact_bundles:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"evalRunId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"evalRunId"
]
}⚪rubrkit_export_proof_report(artifactBundleId, evalRunId, auditRunId, sourceFileVersionId, sourceFileId, ...)
Start an async proof report job with export storage enabled by default. Requires evals:run for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"evalRunId": {
"type": "string",
"format": "uuid"
},
"auditRunId": {
"type": "string",
"format": "uuid"
},
"sourceFileVersionId": {
"type": "string",
"format": "uuid"
},
"sourceFileId": {
"type": "string",
"format": "uuid"
},
"sourceVersionNumber": {
"type": "integer",
"minimum": 1
},
"candidateFileVersionId": {
"type": "string",
"format": "uuid"
},
"candidateFileId": {
"type": "string",
"format": "uuid"
},
"candidateVersionNumber": {
"type": "integer",
"minimum": 1
},
"rubricVersion": {
"type": "string"
},
"includeExport": {
"type": "boolean",
"default": true
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_proof_report(artifactBundleId, proofReportId)
Read one proof report. Requires artifact_bundles:read for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"proofReportId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"proofReportId"
]
}🟢rubrkit_list_golden_cases(artifactBundleId, status)
List the golden cases for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"status": {
"type": [
"string",
"null"
],
"enum": [
"active",
"retired",
null
]
}
},
"required": [
"artifactBundleId"
]
}🟡rubrkit_add_golden_case(artifactBundleId, input, expectedOutput, criteria, origin, ...)
Add a golden case (a real edge-case input with a confirmed-correct output) to an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"input": {
"type": "object"
},
"expectedOutput": {
"type": "object"
},
"criteria": {
"type": "object"
},
"origin": {
"type": "string",
"enum": [
"audit",
"human",
"prod"
]
},
"originRef": {
"type": "object"
},
"weight": {
"type": "integer",
"minimum": 1
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"artifactBundleId",
"input"
]
}🔴rubrkit_retire_golden_case(goldenCaseId)
Retire a golden case via DELETE /api/v1/golden-cases/{goldenCaseId}. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"goldenCaseId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"goldenCaseId"
]
}🟡rubrkit_create_drift_monitor(artifactBundleId, pinnedVersionId, cadence, repeats)
Create a drift monitor pinned to a file version via POST /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"pinnedVersionId": {
"type": "string",
"format": "uuid"
},
"cadence": {
"type": "string"
},
"repeats": {
"type": "integer",
"minimum": 2,
"maximum": 5
}
},
"required": [
"artifactBundleId",
"pinnedVersionId"
]
}🟢rubrkit_list_drift_monitors(artifactBundleId)
List the drift monitors for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_drift_observations(driftMonitorId)
List the recorded observations for a drift monitor via /api/v1/drift-monitors/{driftMonitorId}/observations. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"driftMonitorId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"driftMonitorId"
]
}🟡rubrkit_run_drift_check(driftMonitorId)
Queue a drift check immediately instead of waiting for the monitor's schedule, via POST /api/v1/drift-monitors/{driftMonitorId}/check. The first check on a new monitor calibrates its baseline. Costs one audit per repeat. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"driftMonitorId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"driftMonitorId"
]
}🟡rubrkit_set_drift_monitor_status(driftMonitorId, status)
Pause or resume a drift monitor via PATCH /api/v1/drift-monitors/{driftMonitorId}. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"driftMonitorId": {
"type": "string",
"format": "uuid"
},
"status": {
"type": "string",
"enum": [
"active",
"paused"
]
}
},
"required": [
"driftMonitorId",
"status"
]
}🟡rubrkit_record_outcome(artifactBundleId, artifactVersionId, inputHash, inputFeature, outcomeMetric, ...)
Record a production outcome event (input feature, metric, and/or label) for an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/outcomes. Requires artifact_bundles:write for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"artifactVersionId": {
"type": "string",
"format": "uuid"
},
"inputHash": {
"type": "string"
},
"inputFeature": {
"type": "number"
},
"outcomeMetric": {
"type": "number"
},
"outcomeLabel": {
"type": "string"
},
"occurredAt": {
"type": "string",
"format": "date-time"
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_input_drift(artifactBundleId, windowDays, binCount)
Compare recent production inputs against a prior reference window (population stability index) for an artifact bundle via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/input. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"windowDays": {
"type": "integer",
"minimum": 1
},
"binCount": {
"type": "integer",
"minimum": 1
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_validity_drift(artifactBundleId)
Correlate rubric scores against recorded outcome metrics for an artifact bundle to check whether the rubric is still predictive via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/validity. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Eingabe-Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}Community
Nachweis