rubrkit
MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
Should I use this
Quality & Safety
Findings (5)
- LOWTool 'rubrkit_hard_delete_artifact_bundle' name length outside 3-30 rangein rubrkit_hard_delete_artifact_bundle
- LOWTool 'rubrkit_list_artifact_bundle_versions' name length outside 3-30 rangein rubrkit_list_artifact_bundle_versions
- LOWTool 'rubrkit_restore_artifact_bundle_version' name length outside 3-30 rangein rubrkit_restore_artifact_bundle_version
- LOWTool 'rubrkit_read_drift_observations' name length outside 3-30 rangein rubrkit_read_drift_observations
- LOWTool 'rubrkit_set_drift_monitor_status' name length outside 3-30 rangein rubrkit_set_drift_monitor_status
Based on automated analysis of tool definitions and protocol compliance.
Context Cost
This is the approximate number of tokens consumed each time the server's tools are loaded into a model's context. Higher counts reduce the attention available for other tasks.
Install
One-Click Install
Add this to your `claude_desktop_config.json` file:
{
"mcpServers": {
"rubrkit": {
"url": "https://www.rubrkit.com/api/v1/mcp"
}
}
}Remote endpoints
https://www.rubrkit.com/api/v1/mcpstreamable-httpWhat it can do
Tool inventory
Tools (39)
🟢rubrkit_search_api_docs(query, limit)
Search the canonical docs/api Markdown guides and OpenAPI operation summaries.
Input Schema
{
"type": "object",
"properties": {
"query": {
"type": "string",
"minLength": 1
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 20,
"default": 8
}
},
"required": [
"query"
]
}🟢rubrkit_read_api_doc(slugOrPath)
Read a docs/api Markdown guide or OpenAPI source by slug, file name, or docs/api path.
Input Schema
{
"type": "object",
"properties": {
"slugOrPath": {
"type": "string",
"minLength": 1
}
},
"required": [
"slugOrPath"
]
}🟢rubrkit_read_openapi_contract
Read the canonical docs/api/openapi.yaml contract.
Input Schema
{
"type": "object",
"properties": {}
}🟡rubrkit_create_artifact_bundle(name, description, customRubric, settings)
Create an artifact bundle through /api/v1/artifact-bundles. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"name": {
"type": "string",
"minLength": 1,
"maxLength": 120
},
"description": {
"type": [
"string",
"null"
],
"maxLength": 2000
},
"customRubric": {
"type": "object",
"additionalProperties": true
},
"settings": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"name"
]
}🟢rubrkit_list_artifact_bundles(status, limit)
List artifact bundles through /api/v1/artifact-bundles. Requires artifact_bundles:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"active",
"archived",
"all"
],
"default": "active"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 100
}
}
}🟢rubrkit_read_artifact_bundle(artifactBundleId)
Read artifact bundle details, files, and file tree. Requires artifact_bundles:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🔴rubrkit_delete_artifact_bundle(artifactBundleId)
Archive an artifact bundle (soft delete) through DELETE /api/v1/artifact-bundles/{artifactBundleId}. The bundle moves to the archived state and can be restored or permanently deleted later. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🔴rubrkit_hard_delete_artifact_bundle(artifactBundleId)
Permanently delete an archived artifact bundle and all of its files, versions, and run history through DELETE /api/v1/artifact-bundles/{artifactBundleId}/permanent. This is irreversible and only works on bundles that are already archived. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🟡rubrkit_upload_artifact_files(artifactBundleId, files)
Upload one or more text files into an artifact bundle. Requires files:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"files": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"required": [
"path",
"content"
],
"properties": {
"path": {
"type": "string",
"minLength": 1
},
"content": {
"type": "string"
},
"mediaType": {
"type": "string"
},
"artifactType": {
"type": "string"
},
"message": {
"type": "string",
"maxLength": 500
}
}
}
}
},
"required": [
"artifactBundleId",
"files"
]
}🟢rubrkit_list_artifact_files(artifactBundleId, limit)
List files in an artifact bundle. Requires files:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 200,
"default": 200
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_artifact_file(artifactBundleId, fileId)
Read the latest file metadata and content. Requires files:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"fileId"
]
}🟡rubrkit_update_artifact_file(artifactBundleId, fileId, content, mediaType, message)
Write a new editable version for a file. Requires files:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
},
"content": {
"type": "string"
},
"mediaType": {
"type": "string"
},
"message": {
"type": "string",
"maxLength": 500
}
},
"required": [
"artifactBundleId",
"fileId",
"content"
]
}⚪rubrkit_compress_artifact_file(artifactBundleId, fileId, level, dryRun)
Deterministically shorten a markdown or text prompt without losing instructions (free, no credits). Saves a new version unless dryRun is true. Requires files:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
},
"level": {
"type": "string",
"enum": [
"safe",
"aggressive"
],
"default": "safe"
},
"dryRun": {
"type": "boolean",
"default": false
}
},
"required": [
"artifactBundleId",
"fileId"
]
}⚪rubrkit_start_audit(artifactBundleId, targetFileId, targetVersionNumber, artifactBundleVersionNumber, rubricKey, ...)
Start an async audit job. Audit results are cached per account by a content hash of their inputs; pass force: true to bypass the cache and run a fresh audit (the same as the CLI's --no-cache flag). Requires audits:run for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"targetFileId": {
"type": "string",
"format": "uuid"
},
"targetVersionNumber": {
"type": "integer",
"minimum": 1
},
"artifactBundleVersionNumber": {
"type": "integer",
"minimum": 1
},
"rubricKey": {
"type": "string"
},
"artifactType": {
"type": "string"
},
"customRequirements": {
"type": "array",
"items": {
"type": "string"
}
},
"modelTier": {
"type": "string",
"enum": [
"cheap",
"standard",
"premium"
]
},
"reasoningEffort": {
"type": "string",
"enum": [
"minimal",
"low",
"medium",
"high"
]
},
"force": {
"type": "boolean",
"default": false
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_list_audits(artifactBundleId, limit)
List persisted audit history. Requires artifact_bundles:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 25
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_audit(artifactBundleId, auditRunId)
Read a single audit run. Requires artifact_bundles:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"auditRunId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"auditRunId"
]
}⚪rubrkit_apply_audit_rewrite(artifactBundleId, auditRunId)
Apply a completed audit's AI rewrite to the bundle files (a new version per changed file). Requires audits:run for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"auditRunId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"auditRunId"
]
}🟢rubrkit_poll_job(jobId)
Read async job progress. Requires jobs:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"jobId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"jobId"
]
}🟢rubrkit_list_job_events(jobId, limit)
Read append-only job progress events. Requires jobs:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"jobId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 100
}
},
"required": [
"jobId"
]
}🟢rubrkit_list_artifact_bundle_versions(artifactBundleId, limit)
List bundle-level version events. Requires artifact_bundles:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 100
}
},
"required": [
"artifactBundleId"
]
}⚪rubrkit_restore_artifact_bundle_version(artifactBundleId, versionNumber, message)
Restore files from a prior artifact-bundle version by creating new file versions. Requires artifact_bundles:write and files:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"versionNumber": {
"type": "integer",
"minimum": 1
},
"message": {
"type": "string",
"maxLength": 500
}
},
"required": [
"artifactBundleId",
"versionNumber"
]
}🟢rubrkit_list_file_versions(artifactBundleId, fileId, limit)
List immutable versions for a file. Requires files:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 100
}
},
"required": [
"artifactBundleId",
"fileId"
]
}⚪rubrkit_restore_file_version(artifactBundleId, fileId, versionNumber, message)
Restore a prior file version by creating a new version. Requires files:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"fileId": {
"type": "string",
"format": "uuid"
},
"versionNumber": {
"type": "integer",
"minimum": 1
},
"message": {
"type": "string",
"maxLength": 500
}
},
"required": [
"artifactBundleId",
"fileId",
"versionNumber"
]
}🟡rubrkit_run_evals(artifactBundleId, auditRunId, sourceFileVersionId, sourceFileId, sourceVersionNumber, ...)
Start an async eval run. Requires evals:run for API keys. Set mode: "advanced" with baselineVersionId and candidateVersionId to run a statistically-backed A/B (repeats defaults to 3, max 5).
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"auditRunId": {
"type": "string",
"format": "uuid"
},
"sourceFileVersionId": {
"type": "string",
"format": "uuid"
},
"sourceFileId": {
"type": "string",
"format": "uuid"
},
"sourceVersionNumber": {
"type": "integer",
"minimum": 1
},
"candidateFileVersionId": {
"type": "string",
"format": "uuid"
},
"candidateFileId": {
"type": "string",
"format": "uuid"
},
"candidateVersionNumber": {
"type": "integer",
"minimum": 1
},
"rubricVersion": {
"type": "string"
},
"maxCases": {
"type": "integer",
"minimum": 1,
"maximum": 20
},
"testCases": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"mode": {
"type": "string",
"enum": [
"standard",
"advanced"
]
},
"repeats": {
"type": "integer",
"minimum": 1,
"maximum": 5
},
"baselineVersionId": {
"type": "string",
"format": "uuid"
},
"candidateVersionId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_list_evals(artifactBundleId, limit)
List eval history. Requires artifact_bundles:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 100,
"default": 25
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_eval(artifactBundleId, evalRunId)
Read one eval run, including the `stats` statistical verdict on advanced-mode runs (empty for standard runs). Requires artifact_bundles:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"evalRunId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"evalRunId"
]
}⚪rubrkit_export_proof_report(artifactBundleId, evalRunId, auditRunId, sourceFileVersionId, sourceFileId, ...)
Start an async proof report job with export storage enabled by default. Requires evals:run for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"evalRunId": {
"type": "string",
"format": "uuid"
},
"auditRunId": {
"type": "string",
"format": "uuid"
},
"sourceFileVersionId": {
"type": "string",
"format": "uuid"
},
"sourceFileId": {
"type": "string",
"format": "uuid"
},
"sourceVersionNumber": {
"type": "integer",
"minimum": 1
},
"candidateFileVersionId": {
"type": "string",
"format": "uuid"
},
"candidateFileId": {
"type": "string",
"format": "uuid"
},
"candidateVersionNumber": {
"type": "integer",
"minimum": 1
},
"rubricVersion": {
"type": "string"
},
"includeExport": {
"type": "boolean",
"default": true
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_proof_report(artifactBundleId, proofReportId)
Read one proof report. Requires artifact_bundles:read for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"proofReportId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId",
"proofReportId"
]
}🟢rubrkit_list_golden_cases(artifactBundleId, status)
List the golden cases for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"status": {
"type": [
"string",
"null"
],
"enum": [
"active",
"retired",
null
]
}
},
"required": [
"artifactBundleId"
]
}🟡rubrkit_add_golden_case(artifactBundleId, input, expectedOutput, criteria, origin, ...)
Add a golden case (a real edge-case input with a confirmed-correct output) to an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"input": {
"type": "object"
},
"expectedOutput": {
"type": "object"
},
"criteria": {
"type": "object"
},
"origin": {
"type": "string",
"enum": [
"audit",
"human",
"prod"
]
},
"originRef": {
"type": "object"
},
"weight": {
"type": "integer",
"minimum": 1
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"artifactBundleId",
"input"
]
}🔴rubrkit_retire_golden_case(goldenCaseId)
Retire a golden case via DELETE /api/v1/golden-cases/{goldenCaseId}. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"goldenCaseId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"goldenCaseId"
]
}🟡rubrkit_create_drift_monitor(artifactBundleId, pinnedVersionId, cadence, repeats)
Create a drift monitor pinned to a file version via POST /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"pinnedVersionId": {
"type": "string",
"format": "uuid"
},
"cadence": {
"type": "string"
},
"repeats": {
"type": "integer",
"minimum": 2,
"maximum": 5
}
},
"required": [
"artifactBundleId",
"pinnedVersionId"
]
}🟢rubrkit_list_drift_monitors(artifactBundleId)
List the drift monitors for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_drift_observations(driftMonitorId)
List the recorded observations for a drift monitor via /api/v1/drift-monitors/{driftMonitorId}/observations. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Input Schema
{
"type": "object",
"properties": {
"driftMonitorId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"driftMonitorId"
]
}🟡rubrkit_run_drift_check(driftMonitorId)
Queue a drift check immediately instead of waiting for the monitor's schedule, via POST /api/v1/drift-monitors/{driftMonitorId}/check. The first check on a new monitor calibrates its baseline. Costs one audit per repeat. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"driftMonitorId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"driftMonitorId"
]
}🟡rubrkit_set_drift_monitor_status(driftMonitorId, status)
Pause or resume a drift monitor via PATCH /api/v1/drift-monitors/{driftMonitorId}. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"driftMonitorId": {
"type": "string",
"format": "uuid"
},
"status": {
"type": "string",
"enum": [
"active",
"paused"
]
}
},
"required": [
"driftMonitorId",
"status"
]
}🟡rubrkit_record_outcome(artifactBundleId, artifactVersionId, inputHash, inputFeature, outcomeMetric, ...)
Record a production outcome event (input feature, metric, and/or label) for an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/outcomes. Requires artifact_bundles:write for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"artifactVersionId": {
"type": "string",
"format": "uuid"
},
"inputHash": {
"type": "string"
},
"inputFeature": {
"type": "number"
},
"outcomeMetric": {
"type": "number"
},
"outcomeLabel": {
"type": "string"
},
"occurredAt": {
"type": "string",
"format": "date-time"
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_input_drift(artifactBundleId, windowDays, binCount)
Compare recent production inputs against a prior reference window (population stability index) for an artifact bundle via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/input. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
},
"windowDays": {
"type": "integer",
"minimum": 1
},
"binCount": {
"type": "integer",
"minimum": 1
}
},
"required": [
"artifactBundleId"
]
}🟢rubrkit_read_validity_drift(artifactBundleId)
Correlate rubric scores against recorded outcome metrics for an artifact bundle to check whether the rubric is still predictive via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/validity. Requires artifact_bundles:read (or artifacts:pull) for API keys.
Input Schema
{
"type": "object",
"properties": {
"artifactBundleId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"artifactBundleId"
]
}Community
Evidence