AI PM Lab
Learn how AI products work: lessons, model cost comparisons, real experiments, PM guides.
사용해야 할까요
품질 및 안전성
A
발견 사항 (3)
- LOWTool 'get_lesson' description lacks action verbget_lesson에서
- LOWTool 'define_term' description lacks action verbdefine_term에서
- LOWTool 'prep_planning_meeting' description lacks action verbprep_planning_meeting에서
도구 정의와 프로토콜 준수에 대한 자동 분석을 기반으로 합니다.
컨텍스트 비용
~5,542토큰 (도구 정의)
~2.6 KB일반적인 응답 크기
상당한 주의 영향 (128k 컨텍스트의 4.33%)
이는 서버의 도구가 모델의 컨텍스트에 로드될 때마다 소비되는 대략적인 토큰 수입니다. 수치가 높을수록 다른 작업에 사용할 수 있는 주의가 줄어듭니다.
설치
원클릭 설치
`claude_desktop_config.json` 파일에 다음을 추가하세요:
{
"mcpServers": {
"ai-pm-lab": {
"url": "https://aipmlab.io/api/mcp"
}
}
}원격 엔드포인트
https://aipmlab.io/api/mcpstreamable-http할 수 있는 일
도구 목록
도구 (18)
🟢 읽기 전용🟡 쓰기🔴 삭제⚪ 알 수 없음
🟢search_lessons(query)
Find AI PM Lab lessons on a topic (e.g. RAG, evals, agents, prompt injection, cost). Returns the best matches with their key idea and link.
입력 스키마
{
"type": "object",
"properties": {
"query": {
"type": "string",
"minLength": 1,
"maxLength": 200,
"description": "A topic or question, e.g. 'how do I stop hallucinations'"
}
},
"required": [
"query"
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢get_lesson(slug)
A lesson's key idea, concepts, things to try and suggested prompts, with links to the lesson, its 3D tour and its challenge.
입력 스키마
{
"type": "object",
"properties": {
"slug": {
"type": "string",
"enum": [
"how-llms-work",
"writing-good-prompts",
"hallucination-and-grounding",
"embeddings-and-semantic-search",
"prompting-vs-rag-vs-fine-tuning",
"prompt-caching-and-cost",
"structured-outputs",
"how-ai-agents-work",
"retrieval-augmented-generation",
"tool-design-and-mcp",
"multi-agent-systems",
"workflows-vs-agents",
"ai-evals",
"reasoning-models",
"production-monitoring",
"prompt-injection",
"model-routing",
"human-in-the-loop"
],
"description": "The lesson's slug, from search_lessons"
}
},
"required": [
"slug"
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢list_challenges
The challenges (games scored from real recorded runs, with leaderboards) and what each asks you to do.
입력 스키마
{
"type": "object",
"properties": {},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢define_term(term)
A plain-language definition of an AI product concept (tokens, temperature, RAG, embeddings, LLM-as-judge, prompt injection…), with the lessons that teach it.
입력 스키마
{
"type": "object",
"properties": {
"term": {
"type": "string",
"minLength": 1,
"maxLength": 100,
"description": "The AI term to define, e.g. 'RAG', 'temperature' or 'prompt caching'"
}
},
"required": [
"term"
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢count_tokens(text)
Count the tokens in a text (an estimate with OpenAI's cl100k tokenizer; other model families differ slightly).
입력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"minLength": 1,
"maxLength": 100000,
"description": "The text to count, such as a system prompt or a typical user message"
}
},
"required": [
"text"
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"tokens": {
"type": "number"
},
"characters": {
"type": "number"
},
"tokenizer": {
"type": "string"
}
},
"required": [
"tokens",
"characters",
"tokenizer"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢estimate_cost(model, input_tokens, output_tokens, requests_per_day, cached_share)
What a model call costs per request, per day and per month at a given volume, from AI Gateway's live prices. Model ids look like 'openai/gpt-4.1-mini' or 'anthropic/claude-sonnet-4.5'.
입력 스키마
{
"type": "object",
"properties": {
"model": {
"type": "string",
"minLength": 3,
"maxLength": 100,
"description": "A model id like 'openai/gpt-4.1-mini' or 'anthropic/claude-sonnet-4'"
},
"input_tokens": {
"type": "integer",
"minimum": 0,
"maximum": 2000000,
"description": "Input tokens per request: system prompt, retrieved documents, history and the user's message"
},
"output_tokens": {
"type": "integer",
"minimum": 0,
"maximum": 200000,
"description": "Output tokens per request: the length of a typical reply"
},
"requests_per_day": {
"type": "integer",
"minimum": 1,
"maximum": 100000000,
"description": "Expected requests per day, for the monthly cost"
},
"cached_share": {
"description": "Share of input tokens served from the prompt cache, 0 to 1",
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"required": [
"model",
"input_tokens",
"output_tokens",
"requests_per_day"
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"model": {
"type": "string"
},
"per_request": {
"type": "number"
},
"per_day": {
"type": "number"
},
"per_month": {
"type": "number"
},
"input_share": {
"type": "number",
"description": "Share of the cost that's input tokens, 0 to 1"
},
"cache_saving_per_month": {
"type": "number"
},
"prices_per_million": {
"type": "object",
"properties": {
"input": {
"type": "number"
},
"cached_input": {
"type": "number"
},
"output": {
"type": "number"
}
},
"required": [
"input",
"cached_input",
"output"
],
"additionalProperties": false,
"description": "Dollars per million tokens"
},
"price_source": {
"type": "string",
"enum": [
"catalog",
"fallback"
]
}
},
"required": [
"model",
"per_request",
"per_day",
"per_month",
"input_share",
"cache_saving_per_month",
"prices_per_million",
"price_source"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢compare_models(models, input_tokens, output_tokens, requests_per_day, cached_share)
Compare several models side by side at a given volume: cost per request and per month, prices, context window, output limit, release date and knowledge cutoff, from AI Gateway's live catalog. Pass ids like 'openai/gpt-4.1-mini', or a provider name ('anthropic', 'openai', 'google') for its newest models, so you don't need to know which models are current.
입력 스키마
{
"type": "object",
"properties": {
"models": {
"minItems": 1,
"maxItems": 12,
"type": "array",
"items": {
"type": "string",
"minLength": 2,
"maxLength": 100
},
"description": "Model ids like 'openai/gpt-4.1-mini', or provider names like 'anthropic' for that provider's newest models"
},
"input_tokens": {
"type": "integer",
"minimum": 0,
"maximum": 2000000,
"description": "Input tokens per request: system prompt, retrieved documents, history and the user's message"
},
"output_tokens": {
"type": "integer",
"minimum": 0,
"maximum": 200000,
"description": "Output tokens per request: the length of a typical reply"
},
"requests_per_day": {
"type": "integer",
"minimum": 1,
"maximum": 100000000,
"description": "Expected requests per day, for the monthly cost"
},
"cached_share": {
"description": "Share of input tokens served from the prompt cache, 0 to 1",
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"required": [
"models",
"input_tokens",
"output_tokens",
"requests_per_day"
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"requests_per_day": {
"type": "number"
},
"models": {
"type": "array",
"items": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"per_request": {
"type": "number"
},
"per_month": {
"type": "number"
},
"prices_per_million": {
"type": "object",
"properties": {
"input": {
"type": "number"
},
"cached_input": {
"type": "number"
},
"output": {
"type": "number"
}
},
"required": [
"input",
"cached_input",
"output"
],
"additionalProperties": false,
"description": "Dollars per million tokens"
},
"context_window": {
"type": "number"
},
"max_output": {
"type": "number"
},
"released": {
"type": "string"
},
"knowledge_cutoff": {
"type": "string"
},
"fits": {
"type": "boolean",
"description": "Whether the request's input and output fit the model's context window and output limit"
}
},
"required": [
"model",
"per_request",
"per_month",
"prices_per_million",
"fits"
],
"additionalProperties": false
}
},
"unknown": {
"type": "array",
"items": {
"type": "string"
}
},
"live_catalog": {
"type": "boolean"
}
},
"required": [
"requests_per_day",
"models",
"unknown",
"live_catalog"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢list_experiments(page)
The real runs recorded on one of AI PM Lab's 3D pages or lessons, each described by its setup (e.g. which defences or settings were on). Replay one to see what happened.
입력 스키마
{
"type": "object",
"properties": {
"page": {
"type": "string",
"enum": [
"injection",
"rag",
"evals",
"agent-loop",
"next-token",
"embeddings",
"how-ai-agents-work",
"how-llms-work",
"retrieval-augmented-generation",
"embeddings-and-semantic-search",
"prompt-caching-and-cost",
"multi-agent-systems",
"prompt-injection",
"ai-evals",
"prompting-vs-rag-vs-fine-tuning",
"human-in-the-loop",
"hallucination-and-grounding",
"production-monitoring",
"writing-good-prompts",
"reasoning-models",
"model-routing",
"structured-outputs",
"tool-design-and-mcp",
"workflows-vs-agents"
],
"description": "A 3D page (injection, rag, evals, agent-loop, next-token, embeddings) or a lesson's slug (e.g. prompt-injection, model-routing, hallucination-and-grounding, human-in-the-loop)"
}
},
"required": [
"page"
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢replay_experiment(page, run)
What happened in one recorded real run (e.g. whether a prompt injection leaked data with given defences, or how many answers were made up without a facts sheet): the setup, each step, the result and the model's reply, with a link to run it on the site.
입력 스키마
{
"type": "object",
"properties": {
"page": {
"type": "string",
"enum": [
"injection",
"rag",
"evals",
"agent-loop",
"next-token",
"embeddings",
"how-ai-agents-work",
"how-llms-work",
"retrieval-augmented-generation",
"embeddings-and-semantic-search",
"prompt-caching-and-cost",
"multi-agent-systems",
"prompt-injection",
"ai-evals",
"prompting-vs-rag-vs-fine-tuning",
"human-in-the-loop",
"hallucination-and-grounding",
"production-monitoring",
"writing-good-prompts",
"reasoning-models",
"model-routing",
"structured-outputs",
"tool-design-and-mcp",
"workflows-vs-agents"
],
"description": "A 3D page (injection, rag, evals, agent-loop, next-token, embeddings) or a lesson's slug (e.g. prompt-injection, model-routing, hallucination-and-grounding, human-in-the-loop)"
},
"run": {
"type": "integer",
"minimum": 1,
"maximum": 500,
"description": "The run's number from list_experiments"
}
},
"required": [
"page",
"run"
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢quiz_me(game, round, answer)
A round from AI PM Lab's challenges, built from real recorded runs: Call It (predict what a model did), Spot the Failure (ship or flag a real output) or Which Wins? (pick the better of two setups). Returns the question without its answer; call again with the round id and the user's choice to mark it, see how players on the site called it, and get the next round.
입력 스키마
{
"type": "object",
"properties": {
"game": {
"description": "call-it, spot-the-failure or which-wins; any if left out",
"type": "string",
"enum": [
"call-it",
"spot-the-failure",
"which-wins"
]
},
"round": {
"description": "The id of the round the user is answering, from the last quiz_me result",
"type": "string",
"maxLength": 200
},
"answer": {
"description": "The user's choice for that round: its letter",
"type": "string",
"maxLength": 200
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢review_ai_spec(spec)
Check an AI feature spec or PRD against what AI PM Lab teaches: approach, model and cost, prompt, grounding, output contract, tools, security, human oversight, evals, monitoring, fallbacks. Gaps link to lessons. Returns instructions to follow with the user's text.
입력 스키마
{
"type": "object",
"properties": {
"spec": {
"description": "The spec or PRD text",
"type": "string",
"maxLength": 50000
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢prep_planning_meeting(feature)
The questions a PM should ask engineers about a planned AI feature, with why each matters and what a good answer sounds like. Returns instructions to follow with the user's text.
입력 스키마
{
"type": "object",
"properties": {
"feature": {
"description": "A few sentences about the AI feature",
"type": "string",
"maxLength": 50000
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢design_eval_suite(feature)
Build an evaluation suite for a planned AI feature with AI PM Lab's method: test cases (typical, edge, out of scope, attacks), graders with a judge rubric, pass bars, and a JSON test set. Returns instructions to follow with the user's text.
입력 스키마
{
"type": "object",
"properties": {
"feature": {
"description": "What the AI feature does, who uses it, and what it must never do",
"type": "string",
"maxLength": 50000
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢write_acceptance_criteria(feature)
Acceptance criteria for an AI feature or user story that can actually be tested: Given/When/Then with measurable pass bars on a test set, must-pass safety criteria, latency and cost budgets, and example test cases, ready to paste into a ticket. Returns instructions to follow with the user's text.
입력 스키마
{
"type": "object",
"properties": {
"feature": {
"description": "The AI feature or user story, who uses it, and what it must never do",
"type": "string",
"maxLength": 50000
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢price_ai_feature(feature)
Work out an AI feature's unit economics and pricing: cost per request and per user (light, typical, heavy), levers like caching and routing, pricing options with margins, guardrails, and the assumptions to validate. Returns instructions to follow with the user's text.
입력 스키마
{
"type": "object",
"properties": {
"feature": {
"description": "What the feature does, the model it uses, expected usage, and how the product is priced today",
"type": "string",
"maxLength": 50000
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢plan_ai_launch(feature)
A phased roadmap from where an AI feature is today to launch and its first month live: frame, prototype and model, evals, hardening, pilot, launch, monitoring. Each phase has exit criteria and the tests and checks for this feature, backed by findings from AI PM Lab's recorded real runs (injection, grounding, evals, agent cost). Returns instructions to follow with the user's text.
입력 스키마
{
"type": "object",
"properties": {
"feature": {
"description": "What the AI feature does, who uses it, what it must never do, and where the work stands today",
"type": "string",
"maxLength": 50000
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢review_launch_readiness(plan)
A go/no-go review of an AI feature's launch plan: evals, grounding, attacks, approvals, cost, latency, monitoring, rollout and support, each marked ready, at risk or missing, with blockers and a launch-day watch list. Returns instructions to follow with the user's text.
입력 스키마
{
"type": "object",
"properties": {
"plan": {
"description": "The feature, what was tested, and the launch plan",
"type": "string",
"maxLength": 50000
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}🟢review_ai_incident(incident)
A blameless post-incident review for an AI feature: where it failed (input, retrieval, prompt, model, tools, output checks), why evals and monitoring missed it, actions, and the new eval cases that would catch it next time. Returns instructions to follow with the user's text.
입력 스키마
{
"type": "object",
"properties": {
"incident": {
"description": "What the feature does, what went wrong, and what's known so far",
"type": "string",
"maxLength": 50000
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}출력 스키마
{
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The answer in markdown, as shown to the user; for the guides, instructions to follow with the user's text"
},
"links": {
"type": "array",
"items": {
"type": "string"
},
"description": "The AI PM Lab pages the answer links to, where each idea can be tried hands-on"
}
},
"required": [
"text",
"links"
],
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false
}커뮤니티
증거
최근 관측
검증됨버전이 기록되지 않음도구 18개