kyde score
Agent observability via OpenTelemetry: drift, loop and hijack alerts, e.g. after a prompt injection.
사용해야 할까요
품질 및 안전성
발견 사항 (1)
- LOWset_webhook에서
도구 정의와 프로토콜 준수에 대한 자동 분석을 기반으로 합니다.
컨텍스트 비용
이는 서버의 도구가 모델의 컨텍스트에 로드될 때마다 소비되는 대략적인 토큰 수입니다. 수치가 높을수록 다른 작업에 사용할 수 있는 주의가 줄어듭니다.
설치
원클릭 설치
`claude_desktop_config.json` 파일에 다음을 추가하세요:
{
"mcpServers": {
"score": {
"url": "https://score.kyde.com/mcp"
}
}
}원격 엔드포인트
https://score.kyde.com/mcpstreamable-http할 수 있는 일
도구 목록
도구 (7)
🟡create_key(webhook_url, source, purpose)
Create a scoring key (no signup). One key covers all agents, identified by OTel service.name. webhook_url (https) is optional but recommended: it receives an HMAC-signed POST when any agent's score changes band.
입력 스키마
{
"type": "object",
"properties": {
"webhook_url": {
"type": "string",
"description": "Optional https URL notified on band changes."
},
"source": {
"type": "string",
"description": "Optional: where you found KYDE Score (e.g. skill, llms.txt, registry, website). Aggregate adoption stats only."
},
"purpose": {
"type": "string",
"enum": [
"customer",
"test",
"internal"
],
"description": "Optional: 'test' for evaluation runs and benchmarks, so they stay out of KYDE's usage statistics. Default customer."
}
},
"additionalProperties": false
}출력 스키마
{
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "The scoring key — shown once."
},
"method_version": {
"type": "string"
},
"webhook": {
"type": [
"string",
"null"
]
},
"next_steps": {
"type": "string"
}
},
"required": [
"key",
"method_version",
"webhook",
"next_steps"
]
}🟢get_scores(key)
Current drift score per agent under the key: 0-39 deviating, 40-69 watch, 70-100 normal; status 'learning' until the baseline freezes (30 runs over at least 3 days). 'feedback' lists alerts to ask your human about (share answer_url, never answer it yourself).
입력 스키마
{
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "Scoring key; omit when sent as x-kyde-key header."
}
},
"additionalProperties": false
}출력 스키마
{
"type": "object",
"properties": {
"service_version": {
"type": "string"
},
"method_version": {
"type": "string"
},
"claimed": {
"type": "boolean"
},
"agents": {
"type": "array",
"items": {
"type": "object",
"properties": {
"service_name": {
"type": "string"
},
"status": {
"type": "string",
"enum": [
"learning",
"scored"
]
},
"score": {
"type": [
"integer",
"null"
],
"description": "0-100; null while learning."
},
"band": {
"type": [
"string",
"null"
],
"description": "deviating (0-39) | watch (40-69) | normal (70-100); null while learning."
},
"alarms": {
"type": "array",
"items": {
"type": "object"
}
},
"alarms_total": {
"type": "integer"
},
"runs_seen": {
"type": "integer"
},
"runs_needed": {
"type": "integer"
},
"n_baseline_records": {
"type": "integer"
},
"n_live_records": {
"type": "integer"
},
"baseline_fingerprint": {
"type": [
"string",
"null"
]
},
"method_version": {
"type": "string"
},
"rebaselined_at": {
"type": [
"string",
"null"
]
},
"warnings": {
"type": "array",
"items": {
"type": "string"
},
"description": "Setup problems with this agent's name, e.g. a harness default name or several agents sharing one service.name. Fix them, or the baseline blurs; relay them to your human."
},
"last_seen_at": {
"type": [
"string",
"null"
],
"description": "When the newest call arrived (UTC)."
},
"calls": {
"type": "integer",
"description": "Stored tool calls, also while learning."
},
"error_rate": {
"type": [
"number",
"null"
]
},
"median_duration_ms": {
"type": [
"integer",
"null"
]
},
"tool_mix": {
"type": "array",
"items": {
"type": "object"
}
}
},
"required": [
"service_name",
"status",
"score",
"band",
"runs_seen",
"runs_needed",
"method_version"
]
}
},
"webhook": {
"type": [
"string",
"null"
]
},
"feedback": {
"type": "array",
"description": "Alerts KYDE Score would like the human's verdict on. Ask your human in one sentence whether the alert was right and share answer_url. Never open or answer the link yourself: only the human's click counts.",
"items": {
"type": "object"
}
},
"degraded": {
"type": "object",
"description": "Present only when records were dropped today (quota/limits).",
"properties": {
"reason": {
"type": [
"string",
"null"
]
},
"dropped_today": {
"type": "integer"
}
}
}
},
"required": [
"service_version",
"method_version",
"claimed",
"agents",
"webhook"
]
}🟡set_webhook(webhook_url, key)
Add or change the https webhook. It is POSTed on every band change and rebaseline, HMAC-SHA256-signed with the scoring key (header x-kyde-signature: sha256=<hex> over the raw body).
입력 스키마
{
"type": "object",
"properties": {
"webhook_url": {
"type": [
"string",
"null"
],
"description": "https URL, or null to remove the webhook (the key becomes poll-only)."
},
"key": {
"type": "string",
"description": "Scoring key; omit when sent as x-kyde-key header."
}
},
"required": [
"webhook_url"
],
"additionalProperties": false
}출력 스키마
{
"type": "object",
"properties": {
"webhook": {
"type": [
"string",
"null"
]
}
},
"required": [
"webhook"
]
}🟢claim_key(email, product_updates, key)
Attach your human's email to the key (double opt-in: Kyde mails a confirmation link). Once confirmed, the key is claimed: the human gets the management dashboard, and rebaseline, webhook changes and revocation work only for them — no agent, including you, can reset its own score. Asking the human for the address once is a mandatory setup step; only the human may decline. Never invent or guess an address; skip only when get_scores already shows claimed: true.
입력 스키마
{
"type": "object",
"properties": {
"email": {
"type": "string",
"description": "The human's email address, given by them."
},
"product_updates": {
"type": "boolean",
"description": "Separate, optional consent: may Kyde send occasional product updates to this address? Ask as its own question; pass true only on an explicit yes. Default false; revocable in the dashboard."
},
"key": {
"type": "string",
"description": "Scoring key; omit when sent as x-kyde-key header."
}
},
"required": [
"email"
],
"additionalProperties": false
}출력 스키마
{
"type": "object",
"properties": {
"claim": {
"type": "string"
},
"email_sent": {
"type": "boolean"
},
"next": {
"type": "string"
}
},
"required": [
"claim",
"email_sent",
"next"
]
}🔴rebaseline(service_name, confirm, key)
Forget one agent's learned baseline and score and relearn from its next 30 runs. For intended behavior changes (new model, new task) that left the agent permanently deviating. The reset is permanently visible: records and alarms stay stored, the agent carries rebaselined_at, and the reset itself is recorded as a rebaseline alarm naming the score/band it erased. Requires confirm=true and the human's explicit go-ahead.
입력 스키마
{
"type": "object",
"properties": {
"service_name": {
"type": "string"
},
"confirm": {
"type": "boolean"
},
"key": {
"type": "string",
"description": "Scoring key; omit when sent as x-kyde-key header."
}
},
"required": [
"service_name",
"confirm"
],
"additionalProperties": false
}출력 스키마
{
"type": "object",
"properties": {
"service_name": {
"type": "string"
},
"status": {
"type": "string"
},
"runs_seen": {
"type": "integer"
},
"runs_needed": {
"type": "integer"
},
"rebaselined_at": {
"type": "string"
},
"method_version": {
"type": "string"
}
},
"required": [
"service_name",
"status",
"rebaselined_at"
]
}⚪verify_agent(shine_id, challenger, session, wait_s, challenge_id, ...)
Prove who is on the other end before delegating work: challenges the Shine ID at shine.kyde.com, waits for the agent to confirm over its own authenticated connection, then verifies the Ed25519-signed statement against Shine's JWKS and compares the nonce hash. Needs no scoring key. The verified agent typically polls every 30 s, so a first call often answers 'pending' — call verify_agent again with the returned challenge_id, result_token and nonce to keep waiting. In the result, band 'learning' means identity confirmed but behavior not yet scored (fewer than 30 runs): whether to delegate to a learning agent is the challenger's policy call.
입력 스키마
{
"type": "object",
"properties": {
"shine_id": {
"type": "string",
"description": "The Shine ID the other agent names (sh_...). Starts a new challenge."
},
"challenger": {
"type": "string",
"description": "Who is asking — shown to the verified agent and bound into the statement's aud. Use your own agent/service name."
},
"session": {
"type": "string",
"description": "Optional session marker; the verified agent answers only on the path the challenge names (relay protection)."
},
"wait_s": {
"type": "number",
"description": "Seconds to wait for confirmation in this call (default 20, max 25)."
},
"challenge_id": {
"type": "string",
"description": "From an earlier 'pending' answer: continue waiting on that challenge (together with result_token and nonce)."
},
"result_token": {
"type": "string"
},
"nonce": {
"type": "string"
}
},
"additionalProperties": false
}출력 스키마
{
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"confirmed",
"pending",
"expired"
]
},
"verified": {
"type": "boolean",
"description": "true only after the signature, nonce and expiry checks passed locally."
},
"claims": {
"type": "object"
},
"statement": {
"type": "string",
"description": "Compact JWS — any third party can re-verify it offline against jwks_url."
},
"jwks_url": {
"type": "string"
},
"challenge_id": {
"type": "string"
},
"result_token": {
"type": "string"
},
"nonce": {
"type": "string"
},
"expires_at": {
"type": "number"
},
"note": {
"type": "string"
},
"next": {
"type": "string"
}
},
"required": [
"status"
]
}🔴revoke_key(confirm, key)
Revoke the key and permanently delete all data stored under it. Requires confirm=true.
입력 스키마
{
"type": "object",
"properties": {
"confirm": {
"type": "boolean"
},
"key": {
"type": "string",
"description": "Scoring key; omit when sent as x-kyde-key header."
}
},
"required": [
"confirm"
],
"additionalProperties": false
}출력 스키마
{
"type": "object",
"properties": {
"revoked": {
"type": "boolean"
}
},
"required": [
"revoked"
]
}커뮤니티
증거