SCRIPTMASTERLABS — Calibration Court
Calibration Court MCP: model calibration leaderboard, ledger verify, paid recalibration via x402.
Should I use this
Quality & Safety
Findings (2)
- LOWTool 'court_model_report' description lacks action verbin court_model_report
- LOWTool 'court_calibrate_labels' description lacks action verbin court_calibrate_labels
Based on automated analysis of tool definitions and protocol compliance.
Context Cost
This is the approximate number of tokens consumed each time the server's tools are loaded into a model's context. Higher counts reduce the attention available for other tasks.
Install
One-Click Install
Add this to your `claude_desktop_config.json` file:
{
"mcpServers": {
"calibration-court": {
"url": "https://calibration-court.onrender.com/mcp"
}
}
}Remote endpoints
https://calibration-court.onrender.com/mcpstreamable-httpWhat it can do
Tool inventory
Tools (5)
🟢court_leaderboard(task_type)
Live ranking of decision models by Brier score on resolved real-world questions. Free. Returns empty lists when nothing has resolved.
Input Schema
{
"type": "object",
"properties": {
"task_type": {
"type": "string"
}
}
}🟢court_model_report(model, task_type)
Calibration report for one model: Brier, ECE, reliability bins with Wilson intervals, overconfidence, drift. Free.
Input Schema
{
"type": "object",
"properties": {
"model": {
"type": "string"
},
"task_type": {
"type": "string"
}
},
"required": [
"model"
]
}🟢court_verify_ledger
Recompute the SHA-256 hash chain and cross-check derived tables. Free.
Input Schema
{
"type": "object",
"properties": {}
}🟢court_recalibrate(model, prob, task_type)
Paid (x402, USDC on Base). Map a model's raw probability to an empirically calibrated one using resolved outcomes on the ledger. Refuses with insufficient_data below 30 samples.
Input Schema
{
"type": "object",
"properties": {
"model": {
"type": "string"
},
"prob": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"task_type": {
"type": "string"
}
},
"required": [
"model",
"prob"
]
}🟢court_calibrate_labels(examples, probs)
Paid (x402, USDC on Base). Fit isotonic calibration on YOUR labelled outcomes ({p,y} pairs) and recalibrate the given probabilities. Stateless; nothing is stored.
Input Schema
{
"type": "object",
"properties": {
"examples": {
"type": "array",
"items": {
"type": "object",
"properties": {
"p": {
"type": "number"
},
"y": {
"type": "integer",
"enum": [
0,
1
]
}
},
"required": [
"p",
"y"
]
}
},
"probs": {
"type": "array",
"items": {
"type": "number"
}
}
},
"required": [
"examples",
"probs"
]
}Community
Evidence