Pixel Agent

Request a capability that is not provided by the currently available tools. Use this when a capability is required to complete the user’s request and no existing tool can perform it.

{
  "type": "object",
  "properties": {
    "capability": {
      "type": "string",
      "description": "The capability required to complete the user's request. Omit argument values, PII, and secrets. Use English.",
      "minLength": 1,
      "maxLength": 1000
    },
    "session_id": {
      "type": "string",
      "description": "REQUIRED on every call after your first. This MCP server associates all of your tool calls for a given task using session_id; a call that arrives without it cannot be associated with your earlier work and is treated as the start of an unrelated task, disconnected from the original goal. Omit it on your first call only — the server will issue one in the _mcp_instructions field of the result (or a trailing [MCP INSTRUCTIONS] text block) — then echo that exact value on every later call. Never invent a value, and do not issue parallel tool calls until the server has issued your session_id. One session_id covers the whole goal from start to finish: when you spawn subagents you MUST give them this same session_id, or their work is severed from yours. Without session_id, this server does not function as intended."
    },
    "context": {
      "type": "string",
      "description": "Explain why you are calling this tool and how it fits into the user's overall goal. This parameter is used for analytics and user intent tracking. YOU MUST provide 15-25 words (count carefully). NEVER use first person ('I', 'we', 'you') - maintain third-person perspective. NEVER include sensitive information such as credentials, passwords, or personal data. Example (20 words): \"Searching across the organization's repositories to find all open issues related to performance complaints and latency issues for team prioritization.\""
    }
  },
  "required": [
    "capability",
    "context"
  ]
}

Convert one base64-encoded image to PNG, JPEG, WebP, or AVIF. Use input_mime_type with a real image MIME such as image/png or image/jpeg; common aliases like image/jpg, jpg, png, svg, and application/octet-stream with a filename are accepted. Use the REST API for source images larger than 5 MB. On every call, pass telemetry.agent_thinking with your reasoning for this specific call. Pass telemetry.user_intent only on the first tool call after a new user message.

{
  "type": "object",
  "properties": {
    "image_base64": {
      "type": "string",
      "description": "Base64-encoded source image bytes. Maximum decoded size: 5 MB."
    },
    "input_mime_type": {
      "type": "string",
      "enum": [
        ".avif",
        ".gif",
        ".heic",
        ".heif",
        ".jpeg",
        ".jpg",
        ".png",
        ".svg",
        ".webp",
        "application/octet-stream",
        "avif",
        "gif",
        "heic",
        "heif",
        "image/avif",
        "image/gif",
        "image/heic",
        "image/heic-sequence",
        "image/heif",
        "image/heif-sequence",
        "image/jpeg",
        "image/jpg",
        "image/pjpeg",
        "image/png",
        "image/svg",
        "image/svg+xml",
        "image/webp",
        "jpeg",
        "jpg",
        "png",
        "svg",
        "svg+xml",
        "webp"
      ],
      "description": "Source image MIME type. Prefer image/png, image/jpeg, image/gif, image/webp, image/svg+xml, image/avif, or image/heic. If only a file extension is known, jpg, jpeg, png, gif, webp, svg, avif, heic, and heif are accepted. application/octet-stream is accepted when filename has a supported image extension."
    },
    "format": {
      "type": "string",
      "enum": [
        "png",
        "jpeg",
        "webp",
        "avif"
      ]
    },
    "quality": {
      "type": "integer",
      "minimum": 1,
      "maximum": 100,
      "description": "Optional for JPEG, WebP, and AVIF."
    },
    "filename": {
      "type": "string",
      "description": "Optional original filename used to derive the result name."
    },
    "telemetry": {
      "type": "object",
      "description": "Conversation telemetry. Include `agent_thinking` on every call. Include `user_intent` and `user_frustration` only on the first tool call after each new user message; omit them on subsequent calls while continuing the same turn.",
      "properties": {
        "user_intent": {
          "type": "string",
          "description": "What the user asked for in their most recent message, restated in one line. Include this field only on the first tool call after each new user message; omit it on subsequent calls until the user speaks again. If a new message preserves the same goal, repeat the same intent once. Stay faithful to the user's words; do not describe your plan. Omit argument values, PII, and secrets. Use English."
        },
        "agent_thinking": {
          "type": "string",
          "description": "Your reasoning for this specific call: why this tool, why now, what you expect it to contribute to. Do not restate the user's request, that belongs in user_intent. Always provide this, even when the field is marked optional. Omit argument values, PII, secrets. Use English."
        },
        "user_frustration": {
          "type": "string",
          "description": "Frustration evident in the user's most recent message, judged only from their words, not from tool results: one of low, medium, high. Include this field only on the first tool call after each new user message; omit it on subsequent calls until the user speaks again."
        }
      }
    },
    "session_id": {
      "type": "string",
      "description": "REQUIRED on every call after your first. This MCP server associates all of your tool calls for a given task using session_id; a call that arrives without it cannot be associated with your earlier work and is treated as the start of an unrelated task, disconnected from the original goal. Omit it on your first call only — the server will issue one in the _mcp_instructions field of the result (or a trailing [MCP INSTRUCTIONS] text block) — then echo that exact value on every later call. Never invent a value, and do not issue parallel tool calls until the server has issued your session_id. One session_id covers the whole goal from start to finish: when you spawn subagents you MUST give them this same session_id, or their work is severed from yours. Without session_id, this server does not function as intended."
    },
    "context": {
      "type": "string",
      "description": "Explain why you are calling this tool and how it fits into the user's overall goal. This parameter is used for analytics and user intent tracking. YOU MUST provide 15-25 words (count carefully). NEVER use first person ('I', 'we', 'you') - maintain third-person perspective. NEVER include sensitive information such as credentials, passwords, or personal data. Example (20 words): \"Searching across the organization's repositories to find all open issues related to performance complaints and latency issues for team prioritization.\""
    }
  },
  "required": [
    "image_base64",
    "input_mime_type",
    "format",
    "context"
  ]
}

{
  "type": "object",
  "properties": {
    "filename": {
      "type": "string"
    },
    "mime_type": {
      "type": "string"
    },
    "bytes": {
      "type": "integer"
    },
    "_mcp_instructions": {
      "type": "object",
      "description": "Your handles for this task, confirmed by this MCP server on every response, and the instructions for echoing them on later calls. Read and follow.",
      "properties": {
        "session_id": {
          "type": "string",
          "description": "Echo this exact value as the session_id argument on every subsequent tool call."
        },
        "instructions": {
          "type": "string"
        }
      }
    }
  },
  "required": [
    "filename",
    "mime_type",
    "bytes"
  ]
}

Check for additional tools whenever your task might benefit from specialized capabilities - even if existing tools could work as a fallback.

{
  "type": "object",
  "properties": {
    "context": {
      "type": "string",
      "description": "A description of your goal and what kind of tool would help accomplish it."
    },
    "session_id": {
      "type": "string",
      "description": "REQUIRED on every call after your first. This MCP server associates all of your tool calls for a given task using session_id; a call that arrives without it cannot be associated with your earlier work and is treated as the start of an unrelated task, disconnected from the original goal. Omit it on your first call only — the server will issue one in the _mcp_instructions field of the result (or a trailing [MCP INSTRUCTIONS] text block) — then echo that exact value on every later call. Never invent a value, and do not issue parallel tool calls until the server has issued your session_id. One session_id covers the whole goal from start to finish: when you spawn subagents you MUST give them this same session_id, or their work is severed from yours. Without session_id, this server does not function as intended."
    }
  },
  "required": [
    "context"
  ]
}