Web Search API Eval
Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.
Sollte ich dies verwenden
Qualität und Sicherheit
Befunde (1)
- LOWTool 'fetch' suggests web access but openWorldHint=falsein fetch
Basierend auf einer automatisierten Analyse der Tool-Definitionen und der Einhaltung des Protokolls.
Kontextkosten
Dies ist die ungefähre Anzahl der Tokens, die jedes Mal verbraucht werden, wenn die Tools des Servers in den Kontext eines Modells geladen werden. Höhere Werte verringern die Aufmerksamkeit, die für andere Aufgaben verfügbar ist.
Installieren
Installation mit einem Klick
Fügen Sie dies Ihrer Datei `claude_desktop_config.json` hinzu:
{
"mcpServers": {
"websearch-api-eval": {
"url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
}
}
}Remote-Endpunkte
https://websearch-api-eval.psberkeleymba.workers.dev/mcpstreamable-httpWas es kann
Tool-Inventar
Tools (11)
🟢get_overall_results
Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.
Eingabe-Schema
{
"type": "object",
"properties": {}
}🟢get_scorecard(use_case)
Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.
Eingabe-Schema
{
"type": "object",
"properties": {
"use_case": {
"type": "string",
"enum": [
"overall",
"gtm",
"insurance",
"finance"
],
"description": "overall, gtm, insurance or finance"
}
},
"required": [
"use_case"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢compare_providers(providers, use_case, metric_filter)
Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').
Eingabe-Schema
{
"type": "object",
"properties": {
"providers": {
"minItems": 2,
"type": "array",
"items": {
"type": "string",
"enum": [
"You.com",
"Tavily",
"Exa",
"Parallel"
]
},
"description": "Two or more of: You.com, Tavily, Exa, Parallel"
},
"use_case": {
"default": "overall",
"description": "overall, gtm, insurance or finance",
"type": "string",
"enum": [
"overall",
"gtm",
"insurance",
"finance"
]
},
"metric_filter": {
"description": "Optional keyword to keep only matching metrics",
"type": "string"
}
},
"required": [
"providers"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢recommend_provider(agent_need)
Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.
Eingabe-Schema
{
"type": "object",
"properties": {
"agent_need": {
"description": "What the agent does, e.g. 'voice agent answering in under a second' or 'finance research'",
"type": "string"
}
},
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢list_questions(use_case)
The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.
Eingabe-Schema
{
"type": "object",
"properties": {
"use_case": {
"default": "all",
"description": "all, gtm, insurance or finance",
"type": "string",
"enum": [
"all",
"gtm",
"insurance",
"finance"
]
}
},
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢get_question(id)
One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).
Eingabe-Schema
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Question ID, e.g. G3, I4 or F5"
}
},
"required": [
"id"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢get_charts(chart_id, use_case)
The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.
Eingabe-Schema
{
"type": "object",
"properties": {
"chart_id": {
"description": "e.g. overall_scorecard, finance_key_metrics, gtm_query_grid, time_steps_activation",
"type": "string"
},
"use_case": {
"default": "all",
"description": "Filter the list",
"type": "string",
"enum": [
"all",
"gtm",
"insurance",
"finance",
"setup"
]
}
},
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢get_methodology
How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.
Eingabe-Schema
{
"type": "object",
"properties": {}
}🟢how_to_run_locally
How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.
Eingabe-Schema
{
"type": "object",
"properties": {}
}🟢search(query)
Search the Web Search API Eval benchmark (results, questions, methodology, charts, how to run). Returns matching document IDs, titles and URLs.
Eingabe-Schema
{
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "What to look for"
}
},
"required": [
"query"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}🟢fetch(id)
Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).
Eingabe-Schema
{
"type": "object",
"properties": {
"id": {
"type": "string",
"description": "Document ID from search"
}
},
"required": [
"id"
],
"$schema": "http://json-schema.org/draft-07/schema#"
}Community
Nachweis