SCRIPTMASTERLABS — Calibration Court
Calibration Court MCP: model calibration leaderboard, ledger verify, paid recalibration via x402.
Sollte ich dies verwenden
Qualität und Sicherheit
Befunde (2)
- LOWTool 'court_model_report' description lacks action verbin court_model_report
- LOWTool 'court_calibrate_labels' description lacks action verbin court_calibrate_labels
Basierend auf einer automatisierten Analyse der Tool-Definitionen und der Einhaltung des Protokolls.
Kontextkosten
Dies ist die ungefähre Anzahl der Tokens, die jedes Mal verbraucht werden, wenn die Tools des Servers in den Kontext eines Modells geladen werden. Höhere Werte verringern die Aufmerksamkeit, die für andere Aufgaben verfügbar ist.
Installieren
Installation mit einem Klick
Fügen Sie dies Ihrer Datei `claude_desktop_config.json` hinzu:
{
"mcpServers": {
"calibration-court": {
"url": "https://calibration-court.onrender.com/mcp"
}
}
}Remote-Endpunkte
https://calibration-court.onrender.com/mcpstreamable-httpWas es kann
Tool-Inventar
Tools (5)
🟢court_leaderboard(task_type)
Live ranking of decision models by Brier score on resolved real-world questions. Free. Returns empty lists when nothing has resolved.
Eingabe-Schema
{
"type": "object",
"properties": {
"task_type": {
"type": "string"
}
}
}🟢court_model_report(model, task_type)
Calibration report for one model: Brier, ECE, reliability bins with Wilson intervals, overconfidence, drift. Free.
Eingabe-Schema
{
"type": "object",
"properties": {
"model": {
"type": "string"
},
"task_type": {
"type": "string"
}
},
"required": [
"model"
]
}🟢court_verify_ledger
Recompute the SHA-256 hash chain and cross-check derived tables. Free.
Eingabe-Schema
{
"type": "object",
"properties": {}
}🟢court_recalibrate(model, prob, task_type)
Paid (x402, USDC on Base). Map a model's raw probability to an empirically calibrated one using resolved outcomes on the ledger. Refuses with insufficient_data below 30 samples.
Eingabe-Schema
{
"type": "object",
"properties": {
"model": {
"type": "string"
},
"prob": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"task_type": {
"type": "string"
}
},
"required": [
"model",
"prob"
]
}🟢court_calibrate_labels(examples, probs)
Paid (x402, USDC on Base). Fit isotonic calibration on YOUR labelled outcomes ({p,y} pairs) and recalibrate the given probabilities. Stateless; nothing is stored.
Eingabe-Schema
{
"type": "object",
"properties": {
"examples": {
"type": "array",
"items": {
"type": "object",
"properties": {
"p": {
"type": "number"
},
"y": {
"type": "integer",
"enum": [
0,
1
]
}
},
"required": [
"p",
"y"
]
}
},
"probs": {
"type": "array",
"items": {
"type": "number"
}
}
},
"required": [
"examples",
"probs"
]
}Community
Nachweis