Web Search API Eval
Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.
我該用這個嗎
品質與安全性
發現項目（1）
- LOWTool 'fetch' suggests web access but openWorldHint=false在 fetch 中
根據工具定義與協定合規性的自動化分析。
上下文成本
這是每次將伺服器的工具載入模型上下文時所消耗的約略 token 數量。數量越高，可用於其他工作的注意力就越少。
安裝
一鍵安裝
將以下內容加入你的 `claude_desktop_config.json` 檔案：
{
"mcpServers": {
"websearch-api-eval": {
"url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
}
}
}遠端端點
https://websearch-api-eval.psberkeleymba.workers.dev/mcpstreamable-http它能做什麼
工具清單
工具（11）
🟢get_overall_results
Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.
輸入結構描述
{
"type": "object",
"properties": {}
}🟢get_scorecard(use_case)
Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.
輸入結構描述
{
"type": "object",
"properties": {
"use_case": {
"type": "string",
"enum": [
"overall",
"gtm",
"insurance",
"finance"
],
"description": "overall, gtm, insurance or finance"
}
},
"required": [
"use_case"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢compare_providers(providers, use_case, metric_filter)
Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').
輸入結構描述
{
"type": "object",
"properties": {
"providers": {
"minItems": 2,
"type": "array",
"items": {
"type": "string",
"enum": [
"You.com",
"Tavily",
"Exa",
"Parallel"
]
},
"description": "Two or more of: You.com, Tavily, Exa, Parallel"
},
"use_case": {
"default": "overall",
"description": "overall, gtm, insurance or finance",
"type": "string",
"enum": [
"overall",
"gtm",
"insurance",
"finance"
]
},
"metric_filter": {
"description": "Optional keyword to keep only matching metrics",
"type": "string"
}
},
"required": [
"providers"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢recommend_provider(agent_need)
Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.
輸入結構描述
{
"type": "object",
"properties": {
"agent_need": {
"description": "What the agent does, e.g. 'voice agent answering in under a second' or 'finance research'",
"type": "string"
}
},
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢list_questions(use_case)
The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.
輸入結構描述
{
"type": "object",
"properties": {
"use_case": {
"default": "all",
"description": "all, gtm, insurance or finance",
"type": "string",
"enum": [
"all",
"gtm",
"insurance",
"finance"
]
}
},
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢get_question(id)
One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).
輸入結構描述
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Question ID, e.g. G3, I4 or F5"
}
},
"required": [
"id"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢get_charts(chart_id, use_case)
The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.
輸入結構描述
{
"type": "object",
"properties": {
"chart_id": {
"description": "e.g. overall_scorecard, finance_key_metrics, gtm_query_grid, time_steps_activation",
"type": "string"
},
"use_case": {
"default": "all",
"description": "Filter the list",
"type": "string",
"enum": [
"all",
"gtm",
"insurance",
"finance",
"setup"
]
}
},
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢get_methodology
How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.
輸入結構描述
{
"type": "object",
"properties": {}
}🟢how_to_run_locally
How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.
輸入結構描述
{
"type": "object",
"properties": {}
}🟢search(query)
Search the Web Search API Eval benchmark (results, questions, methodology, charts, how to run). Returns matching document IDs, titles and URLs.
輸入結構描述
{
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "What to look for"
}
},
"required": [
"query"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢fetch(id)
Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).
輸入結構描述
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Document ID from search"
}
},
"required": [
"id"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}社群
證據