framesail
Create long-form YouTube videos end to end: script, storyboard, voiceover, final MP4.
Should I use this
Quality & Safety
Findings (1)
- LOWTool 'projects_read' description lacks action verbin projects_read
Based on automated analysis of tool definitions and protocol compliance.
Context Cost
This is the approximate number of tokens consumed each time the server's tools are loaded into a model's context. Higher counts reduce the attention available for other tasks.
Install
One-Click Install
Add this to your `claude_desktop_config.json` file:
{
"mcpServers": {
"framesail": {
"url": "https://api.framesail.com/mcp"
}
}
}Remote endpoints
https://api.framesail.com/mcpstreamable-httpWhat it can do
Tool inventory
Tools (28)
🟢get_status(project_id, channel_id, wait)
Where things stand and what to do next; start and resume here. Always reports the account: plan, credits, credits held by running jobs, and credits_available to spend. project_id adds the project's settings, each step's state and web link, jobs with readable errors, export_url, next_step and, when something failed, failures: the last failed jobs and the failed segments grouped by kind and error. channel_id reports the channel style instead: analysis_done, inputs_stale, templates, templates_stale and next_step. Without API access it reports only the account and subscription_needed: pass that on to the user.
Input Schema
{
"type": "object",
"properties": {
"project_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Project id"
},
"channel_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Report this channel's style instead of a project"
},
"wait": {
"default": false,
"description": "Wait up to about 50 seconds for the jobs in scope to finish, then report only credits, jobs, step states and next_step; call again if some still run",
"type": "boolean"
}
}
}🟢view_image(url)
Show a Framesail image inline: a segment image, reference image, style template or preset preview.
Input Schema
{
"type": "object",
"properties": {
"url": {
"description": "An image URL from a read tool (public_url, reference_image, template or preset); other hosts are refused",
"type": "string"
}
},
"required": [
"url"
]
}🟢get_generation_options(generation, action, provider, talking_head)
What a generate call can be told to use. models: the allowed models with credit estimates and settings_schema (the valid settings keys); is_default marks the default (the account's saved choice, else Framesail's). Voice and clip models come in families, each with its own default: pass provider or talking_head for one family. prompt_sections: sections with entity_backed=false can be overridden with editable_sections; entity_backed=true sections show the item's own content (description, prompts, text, brief): change it with the edit tools.
Input Schema
{
"type": "object",
"properties": {
"generation": {
"description": "script_scan is Analyze Script; *_reference, reference images; voice_block, the voiceover; segment_image, a segment's image; segment_video, a clip; art_style_*_template, the style's template images",
"enum": [
"script",
"script_scan",
"storyboard",
"character_reference",
"environment_reference",
"object_reference",
"voice_block",
"segment_image",
"segment_video",
"art_style_character_template",
"art_style_environment_template"
],
"type": "string"
},
"action": {
"description": "models: the allowed models; prompt_sections: the prompt editor's sections",
"enum": [
"models",
"prompt_sections"
],
"type": "string"
},
"provider": {
"anyOf": [
{
"enum": [
"minimax",
"elevenlabs"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "voice_block: the voice engine (the project's voice_tts_provider)"
},
"talking_head": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"description": "segment_video: true for talking head models, false for clips"
}
},
"required": [
"generation",
"action"
]
}🔴account_edit(action, generation, model, settings, provider, ...)
The account's default model and settings for one kind of generation, used when neither the call nor the item's prompt editor names one. voice_block needs provider and segment_video needs talking_head: each family keeps its own default.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "set_model_default: model, optional settings; clear_model_default",
"enum": [
"set_model_default",
"clear_model_default"
],
"type": "string"
},
"generation": {
"description": "script_scan is Analyze Script; *_reference, reference images; voice_block, the voiceover; segment_image, a segment's image; segment_video, a clip; art_style_*_template, the style's template images",
"enum": [
"script",
"script_scan",
"storyboard",
"character_reference",
"environment_reference",
"object_reference",
"voice_block",
"segment_image",
"segment_video",
"art_style_character_template",
"art_style_environment_template"
],
"type": "string"
},
"model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "set_model_default: a model id from get_generation_options"
},
"settings": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Model settings: keys of its settings_schema"
},
"provider": {
"anyOf": [
{
"enum": [
"minimax",
"elevenlabs"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "voice_block: the voice engine (the project's voice_tts_provider)"
},
"talking_head": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"description": "segment_video: true for talking head models, false for clips"
}
},
"required": [
"action",
"generation"
]
}🔴delete(type, project_id, id, segment_number)
Permanently delete; irreversible, so confirm with the user first. project: it and everything in it. asset: a character, environment or object. music_track: a background music track. segment_extra: a segment's sound effect or overlay, with its scene layers. A segment's image, clip and voiceover can't be deleted.
Input Schema
{
"type": "object",
"properties": {
"type": {
"description": "project: project_id; asset, music_track: id; segment_extra: project_id, segment_number, and id if the segment has several",
"enum": [
"project",
"asset",
"music_track",
"segment_extra"
],
"type": "string"
},
"project_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Project id"
},
"id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "The asset, music track, or segment sound effect or overlay"
},
"segment_number": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"description": "Segment number (1-based, from storyboard_read)"
}
},
"required": [
"type"
]
}🟢projects_read(action, channel_id)
Your channels, or the projects in one channel.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "channels: your channels; projects: the projects in channel_id",
"enum": [
"channels",
"projects"
],
"type": "string"
},
"channel_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "projects: the channel"
}
},
"required": [
"action"
]
}🔴projects_edit(action, name, channel_id, project_id, title, ...)
Create a channel or a project, or change a project's title, brief or video format. A new project takes its channel's style; a channel with no style needs one (style_edit action=create) before anything is generated. Share the returned web_url so the user can follow along.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "create_channel: name, optional description; create_project: channel_id, title, optional description, video_format; update_project: project_id and any of title, description, video_format",
"enum": [
"create_channel",
"create_project",
"update_project"
],
"type": "string"
},
"name": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "create_channel: the channel name"
},
"channel_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "create_project: its channel"
},
"project_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Project id"
},
"title": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "The project title"
},
"description": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "A channel: what it is about. A project: the brief, what the video is about; it seeds the script, so make it specific"
},
"video_format": {
"anyOf": [
{
"enum": [
"landscape",
"portrait"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "landscape (16:9, the default) or portrait (9:16: Shorts, Reels, TikTok); fixed once a storyboard exists"
}
},
"required": [
"action"
]
}🟢style_read(action, channel_id, project_id, include_script_prompt)
get: the channel style: its references, its fields (art_style, narrative_style, director_style, text_style, any added; script_prompt on request), the summary analysis wrote for each, and templates, the character and environment template images (null: not rendered; a URL: it exists, so view_image it rather than render it again). inputs_stale: the references changed since the last analysis (offer style_generate action=analyze). templates_stale: the art style or its image references changed since the templates were rendered (offer to render them again). presets: the preset catalog, one list per axis (art_style, narrative_style, director_style), with labels, descriptions and, for art presets, preview images. Let the user pick one per axis.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "get: the channel style (channel_id or project_id); presets: the preset catalog",
"enum": [
"get",
"presets"
],
"type": "string"
},
"channel_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "get: the channel (or pass project_id)"
},
"project_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Project id"
},
"include_script_prompt": {
"default": false,
"description": "get: include the long script_prompt",
"type": "boolean"
}
},
"required": [
"action"
]
}🔴style_edit(action, channel_id, name, references, presets, ...)
Create or change the channel style (one per channel, used by all its projects). create: offer the user two real options, never invented ones. (a) References (best): YouTube links of their channel or of videos whose look they like, or a written description. Analysis starts at once and writes the fields: wait with get_status channel_id wait=true, then render both template images (style_generate action=template). A reference informs every axis, narrative included; for its look only, give it contributes_to ["art_style"]. (b) Presets: one per axis, all three; instant, and the art preset brings its template images. Presets and references don't mix. update_fields: replace fields' text; it never starts an analysis (an analysis rewrites art_style, narrative_style, director_style and text_style). Those four and script_prompt can't be removed, every field needs text, and summaries are written only by analysis. Real photos and B-roll are turned on by the literal `@real-media` tag in director_style, with prose on when and how much; remove it to turn them off. update_references: replace the references, fields untouched (inputs_stale turns true). File references can only be added in the app.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "create: name, and references or presets; update_fields: fields; update_references: references",
"enum": [
"create",
"update_fields",
"update_references"
],
"type": "string"
},
"channel_id": {
"description": "The channel whose style this is",
"type": "string"
},
"name": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "create: the style name"
},
"references": {
"anyOf": [
{
"items": {
"additionalProperties": true,
"type": "object"
},
"type": "array"
},
{
"type": "null"
}
],
"description": "[{\"input_type\": \"youtube\" | \"text\", \"value\": \"<link or description>\", \"contributes_to\": [\"art_style\", ...]?}]. A youtube value is a video link, or a channel link or @handle. update_references: the COMPLETE list in order; copy kept entries from style_read verbatim."
},
"presets": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "create: {\"art_style\": id, \"narrative_style\": id, \"director_style\": id} from style_read action=presets"
},
"fields": {
"anyOf": [
{
"additionalProperties": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"type": "object"
},
{
"type": "null"
}
],
"description": "update_fields: {key: \"new text\"} for the keys you change; {key: null} removes a key you added"
}
},
"required": [
"action",
"channel_id"
]
}🔴style_generate(action, channel_id, template_type, replace, model, ...)
analyze: read the references again and rewrite art_style, narrative_style, director_style and text_style, replacing hand edits. template: render one of the style's two template images. Reference images render on them (characters on the character template, environments and objects on the environment one) and segment images fall back on them, so finish both before any reference image. An existing one is refused unless replace=true: replacing it changes the look of everything rendered after.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "analyze; template: template_type",
"enum": [
"analyze",
"template"
],
"type": "string"
},
"channel_id": {
"description": "The channel whose style this is",
"type": "string"
},
"template_type": {
"anyOf": [
{
"enum": [
"character",
"environment"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "template: which template image; run once for each"
},
"replace": {
"default": false,
"description": "template: true only when the user asked for a new image (or agreed to render a stale one again)",
"type": "boolean"
},
"model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Model id (get_generation_options); omit for the saved choice"
},
"settings": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Model settings: keys of its settings_schema"
},
"editable_sections": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Template prompt section overrides, keyed by section id"
}
},
"required": [
"action",
"channel_id"
]
}🟢script_read(project_id)
The active script, its versions, and the script settings (script_target_minutes, voice_mix, visual concept). Show the script to the user before Analyze Script.
Input Schema
{
"type": "object",
"properties": {
"project_id": {
"description": "Project id",
"type": "string"
}
},
"required": [
"project_id"
]
}🔴script_edit(action, project_id, content, script_id, instruction, ...)
Change the script or its settings; only revise spends credits. save: replace the active version's text (the old text is lost: show the user the current script first), then run Analyze Script. Notation, never read aloud: `[SCENE: <visual direction>]` as its own paragraph directs the storyboard for the narration after it, until the next one, over the director style. `<break time="1.5s" />` is the only thing that makes the voice pause (about 0.5s between thoughts, 1.0s between sections; one tag with the total, never two in a row). A paragraph starting `Name: "..."` is that character's dialogue. A delivery cue in square brackets inside a line (`[laughs]`, `[sighs]`, `[whispers]`) shapes how it is said; keep it inline, never on its own line. MiniMax performs sounds such as laughs, sighs, breathes, gasps and coughs and drops other cues. activate_version: switch to another version (each generated script is one). revise: an AI rewrite of a passage or the whole script in the narrative voice; returns revised_text without saving it: splice it in and save.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "save: content; activate_version: script_id; revise: instruction, optional selected_text; settings: script_target_minutes and/or voice_mix; set_concept: visual_concept",
"enum": [
"save",
"activate_version",
"revise",
"settings",
"set_concept"
],
"type": "string"
},
"project_id": {
"description": "Project id",
"type": "string"
},
"content": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "save: the full script text"
},
"script_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "activate_version: a version id from script_read"
},
"instruction": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "revise: what to change, e.g. \"make the intro punchier\""
},
"selected_text": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "revise: the exact passage to rewrite; omit for the whole script"
},
"script_target_minutes": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"description": "settings: length of the next generated script, in minutes"
},
"voice_mix": {
"anyOf": [
{
"enum": [
"narrator_only",
"narrator_led",
"balanced",
"character_led"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "settings: narration versus character dialogue"
},
"visual_concept": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "set_concept: how the video looks"
}
},
"required": [
"action",
"project_id"
]
}🔴script_generate(action, project_id, model, settings, editable_sections)
generate: write the script from the brief and the narrative style; each run is a new version (review it with script_read). analyze (Analyze Script): find the script's characters, environments and objects (including any named only in `[SCENE: ...]` directions) and split it into voice blocks. Existing assets are kept as they are (description, reference image, voice) and new ones added; assets the script no longer names stay until the user deletes them. Review with assets_read.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "generate: write the script; analyze: Analyze Script",
"enum": [
"generate",
"analyze"
],
"type": "string"
},
"project_id": {
"description": "Project id",
"type": "string"
},
"model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Model id (get_generation_options); omit for the saved choice"
},
"settings": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Model settings: keys of its settings_schema"
},
"editable_sections": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Template prompt section overrides, keyed by section id"
}
},
"required": [
"action",
"project_id"
]
}🟢assets_read(project_id, asset_type)
The project's characters, environments and objects, each with its description (what every render follows), reference_image (null: not rendered; view_image it) and, for characters, voice_id. Each needs a reference image before the voiceover.
Input Schema
{
"type": "object",
"properties": {
"project_id": {
"description": "Project id",
"type": "string"
},
"asset_type": {
"anyOf": [
{
"enum": [
"character",
"environment",
"object"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "Only this kind"
}
},
"required": [
"project_id"
]
}🔴assets_edit(action, project_id, asset_id, asset_type, name, ...)
Add a character, environment or object that Analyze Script missed, or rename one or rewrite its description. After a description change, render its reference image again (assets_generate asset_ids).
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "create: project_id, asset_type, name, optional description; update: asset_id and name and/or description",
"enum": [
"create",
"update"
],
"type": "string"
},
"project_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Project id"
},
"asset_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "update: the asset id"
},
"asset_type": {
"anyOf": [
{
"enum": [
"character",
"environment",
"object"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "create: the kind"
},
"name": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "The name the script uses for it"
},
"description": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "What it looks like; every render follows it, so be specific"
}
},
"required": [
"action"
]
}🔴assets_generate(project_id, asset_ids, dry_run, model, settings, ...)
Render reference images, which keep each character, environment and object looking the same in every segment. Needs both style template images. Without asset_ids it renders every asset with none, skipping ones already rendering, so a repeat never bills twice. Afterwards, view_image them to check likeness.
Input Schema
{
"type": "object",
"properties": {
"project_id": {
"description": "Project id",
"type": "string"
},
"asset_ids": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "null"
}
],
"description": "Assets to render, again if they have one; omit for every asset without a reference image"
},
"dry_run": {
"default": false,
"description": "Return the count and credit estimate; start nothing",
"type": "boolean"
},
"model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Model id (get_generation_options); omit for the saved choice"
},
"settings": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Model settings: keys of its settings_schema"
},
"editable_sections": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Template prompt section overrides, keyed by section id"
}
},
"required": [
"project_id"
]
}🟢voiceover_read(action, project_id, provider, group, include_subtitle_data)
voices: one engine's voice ids, labels and preview audio, as {groups, voices}. blocks: the voice blocks (narration and dialogue) with their speaker, voice_id (the speaker's voice), audio and any `[SCENE: ...]` direction over them (never spoken), as {blocks, total_duration_ms, all_complete}.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "voices: the voice catalog for provider; blocks: the project's voice blocks",
"enum": [
"voices",
"blocks"
],
"type": "string"
},
"project_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Project id"
},
"provider": {
"default": "minimax",
"description": "voices: the voice engine (the project's voice_tts_provider)",
"enum": [
"minimax",
"elevenlabs"
],
"type": "string"
},
"group": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "voices: one catalog group; each engine has its own (see groups). Filter the large ElevenLabs catalog."
},
"include_subtitle_data": {
"default": false,
"description": "blocks: include word-level timings (bulky)",
"type": "boolean"
}
},
"required": [
"action"
]
}🔴voiceover_edit(action, project_id, speaker, voice_id, voice_tts_provider, ...)
Voices and voice blocks; free. set_voice: the narrator's or a character's voice; every speaker needs one before the voiceover. rescan_blocks: split the active script into voice blocks again after an edit, assets untouched; blocks whose words are unchanged keep their audio. settings: the voice engine, narrator speed and language.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "set_voice: speaker, voice_id; rescan_blocks; settings: voice_tts_provider, narrator_speed and/or language",
"enum": [
"set_voice",
"rescan_blocks",
"settings"
],
"type": "string"
},
"project_id": {
"description": "Project id",
"type": "string"
},
"speaker": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "set_voice: \"narrator\", or a character's name"
},
"voice_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "set_voice: a voice id from voiceover_read action=voices"
},
"voice_tts_provider": {
"anyOf": [
{
"enum": [
"minimax",
"elevenlabs"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "settings: the voice engine"
},
"narrator_speed": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"description": "settings: speaking rate, 0.5-2.0 (default 1.0); pauses come only from `<break>` tags"
},
"language": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "settings: the script's language as an ISO 639-1 code; Analyze Script detects it and overwrites it on every run"
}
},
"required": [
"action",
"project_id"
]
}🔴voiceover_generate(project_id, voice_block_ids, dry_run, model, settings, ...)
Generate voice block audio. Every speaker needs a voice first, and every asset a reference image. Without voice_block_ids it fills every block with no audio, skipping ones already generating. A model must match the project's voice engine. An existing storyboard retimes to the new audio on its own.
Input Schema
{
"type": "object",
"properties": {
"project_id": {
"description": "Project id",
"type": "string"
},
"voice_block_ids": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "null"
}
],
"description": "Blocks to generate, again if they have audio; omit for every block without audio"
},
"dry_run": {
"default": false,
"description": "Return the count and credit estimate; start nothing",
"type": "boolean"
},
"model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Model id (get_generation_options); omit for the saved choice"
},
"settings": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Model settings: keys of its settings_schema"
},
"editable_sections": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Template prompt section overrides, keyed by section id"
}
},
"required": [
"project_id"
]
}🟢storyboard_read(action, project_id, segment_number, offset, limit)
segments: {total, offset, returned, segments}. Each segment has its segment_number (the handle segment tools take), kind (image, clip, talking head, real photo, B-roll or overlay segment), narration, duration, direction, image_prompt, video_prompt (the motion when animated), media_queries (searches for a real photo or B-roll), the characters, environments and objects it shows, continues_from_segment (its image continues that earlier segment's image) and its assets. segment_assets: {segment_number, assets}: its image, clip, real photos, overlays and sound effects, each with kind, status, prompt, model, earlier_renders (what a rollback can restore) and public_url (for view_image).
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "segments: the storyboard, paged; segment_assets: one segment's assets (segment_number)",
"enum": [
"segments",
"segment_assets"
],
"type": "string"
},
"project_id": {
"description": "Project id",
"type": "string"
},
"segment_number": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"description": "Segment number (1-based, from storyboard_read)"
},
"offset": {
"default": 0,
"description": "segments: first to return, 0-based",
"type": "integer"
},
"limit": {
"default": 0,
"description": "segments: how many; 0 returns all. Long videos have 100+, so page.",
"type": "integer"
}
},
"required": [
"action",
"project_id"
]
}🔴storyboard_edit(action, project_id, sfx_level)
Storyboard settings, used by the next storyboard run.
Input Schema
{
"type": "object",
"properties": {
"action": {
"const": "settings",
"description": "settings: sfx_level",
"type": "string"
},
"project_id": {
"description": "Project id",
"type": "string"
},
"sfx_level": {
"description": "How many sound effects the storyboard plans",
"enum": [
"none",
"minimal",
"frequent"
],
"type": "string"
}
},
"required": [
"action",
"project_id",
"sfx_level"
]
}🔴storyboard_generate(project_id, model, settings, editable_sections)
Plan the storyboard: every segment, its timing, image and motion prompts, overlays and continuations, following the channel style and any `[SCENE: ...]` directions. Needs the finished voiceover (its timing drives the cuts). It plans images, plus real photos and B-roll when the director style has the `@real-media` tag; clips come later, by animating rendered images. Running it again replaces the whole plan, edits included. The longest step (several minutes): get_status wait=true, repeatedly.
Input Schema
{
"type": "object",
"properties": {
"project_id": {
"description": "Project id",
"type": "string"
},
"model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Model id (get_generation_options); omit for the saved choice"
},
"settings": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Model settings: keys of its settings_schema"
},
"editable_sections": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Template prompt section overrides, keyed by section id"
}
},
"required": [
"project_id"
]
}🔴segment_edit(action, project_id, segment_number, image_prompt, video_prompt, ...)
Change one segment without losing renders; only split spends credits. Returns the changed segments. set_prompts: write prompts verbatim, no AI rewrite. The renders stay as they are: render the segment again to apply the new prompts. set_continuation: make this segment's image continue an earlier one's (same composition, only the change); render the image again. rollback: restore one of the last 5 renders of the image or clip; the current one takes its place, so it can be undone, and the prompt editor's model stays as it is. split: cut into 2-4 parts; the inheriting part keeps the prompts and renders; the others start empty and get prompts written from their narration in the background (spends credits), or give them prompts with set_prompts. Later segments renumber: read the storyboard again before more edits.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "set_prompts: image_prompt, video_prompt and/or media_queries; set_continuation: continues_from; rollback: what, optional index; split: offsets_ms, optional inherit_index",
"enum": [
"set_prompts",
"set_continuation",
"rollback",
"split"
],
"type": "string"
},
"project_id": {
"description": "Project id",
"type": "string"
},
"segment_number": {
"description": "Segment number (1-based, from storyboard_read)",
"type": "integer"
},
"image_prompt": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "set_prompts: the image prompt"
},
"video_prompt": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "set_prompts: the motion used when the image is animated"
},
"media_queries": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "null"
}
],
"description": "set_prompts: searches for a real photo or B-roll"
},
"continues_from": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"description": "set_continuation: an EARLIER segment number; 0 clears the link"
},
"what": {
"anyOf": [
{
"enum": [
"image",
"clip"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "rollback: restore the image or the clip"
},
"index": {
"default": 0,
"description": "rollback: which earlier render, 0 = the latest",
"type": "integer"
},
"offsets_ms": {
"anyOf": [
{
"items": {
"type": "integer"
},
"type": "array"
},
{
"type": "null"
}
],
"description": "split: 1-3 ascending cut points, in ms from the segment's start; word timings: voiceover_read action=blocks include_subtitle_data=true"
},
"inherit_index": {
"default": 0,
"description": "split: the part (0-based) that keeps the prompts and renders",
"type": "integer"
}
},
"required": [
"action",
"project_id",
"segment_number"
]
}🔴segment_restructure(action, project_id, segment_number, with_segment_number, keep, ...)
Change what a segment is. Every call previews first: when no rendered image or clip would be lost it runs; otherwise it changes nothing and returns what would be lost. Show the user, then call again with confirm=true. combine: merge with a neighboring segment. The kept segment's renders stay, retimed to the combined length, but go stale: its prompts are rewritten for the combined narration, so render it again after. The other's are deleted (the preview names it). change_type: to an image, real photo, B-roll (both fetched) or overlay segment. Clips and talking heads come from render action=animate. keep_image turns a clip back into its image, keeping the render. A change between an image, real photo and B-roll keeps the prompts; render the image again. rewrite: an AI rewrite of the prompts from feedback; render the image and clip again after. For exact text, use segment_edit set_prompts. combine, rewrite and a change to or from an overlay segment write prompts in the background, and a change to a real photo or B-roll with no searches yet writes its searches (its prompts stay): a small AI step that spends credits (get_status wait=true). Until it finishes, change_type is refused (409). Sound effects and overlays always survive; change an overlay's text with editor_edit scene_note. Later segments may renumber: read the storyboard again.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "combine: with_segment_number, optional keep; change_type: to, optional keep_image; rewrite: feedback",
"enum": [
"combine",
"change_type",
"rewrite"
],
"type": "string"
},
"project_id": {
"description": "Project id",
"type": "string"
},
"segment_number": {
"description": "Segment number (1-based, from storyboard_read)",
"type": "integer"
},
"with_segment_number": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"description": "combine: the neighboring segment"
},
"keep": {
"default": "this",
"description": "combine: whose prompts and renders survive",
"enum": [
"this",
"other"
],
"type": "string"
},
"to": {
"anyOf": [
{
"enum": [
"image",
"real_photo",
"b_roll",
"overlay_segment"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "change_type: the new kind"
},
"keep_image": {
"default": false,
"description": "change_type from a clip to image: keep the rendered image",
"type": "boolean"
},
"feedback": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "rewrite: what to change, e.g. \"make this a close-up\""
},
"confirm": {
"default": false,
"description": "Run even though renders are lost; only after the user saw which",
"type": "boolean"
}
},
"required": [
"action",
"project_id",
"segment_number"
]
}🔴render(action, project_id, segment_numbers, what, dry_run, ...)
Render segment images and clips, the biggest spend in the video: always run it with dry_run=true first and get the user's yes on the estimate. generate: without segment_numbers, everything not yet rendered (items rendering are skipped); with them, those segments, finished renders included, e.g. the first minute before the whole video. Overrides need segment_numbers and what=images or what=clips. animate: turn one segment's rendered image into a clip; animate only images worth motion. dry_run quotes the default clip or talking head model. A refused render leaves the segment as it was; to go back to the image, segment_restructure change_type to=image keep_image=true. Check renders with storyboard_read action=segment_assets and view_image.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "generate: optional segment_numbers, what; animate: segment_number, optional talking_head",
"enum": [
"generate",
"animate"
],
"type": "string"
},
"project_id": {
"description": "Project id",
"type": "string"
},
"segment_numbers": {
"anyOf": [
{
"items": {
"type": "integer"
},
"type": "array"
},
{
"type": "null"
}
],
"description": "generate: only these segments, finished ones included; omit for everything not yet rendered"
},
"what": {
"default": "all",
"description": "generate: images (everything but clips: images, overlay segments, real photos, B-roll), clips, or all",
"enum": [
"images",
"clips",
"all"
],
"type": "string"
},
"dry_run": {
"default": false,
"description": "Return the count and credit estimate; start nothing",
"type": "boolean"
},
"segment_number": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"description": "Segment number (1-based, from storyboard_read)"
},
"talking_head": {
"default": false,
"description": "animate: true for a talking head (the person in the image speaks the narration with lip sync, billed on the segment's full length); false for a clip",
"type": "boolean"
},
"model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Model id (get_generation_options); omit for the saved choice"
},
"settings": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Model settings: keys of its settings_schema"
},
"editable_sections": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "Template prompt section overrides, keyed by section id"
}
},
"required": [
"action",
"project_id"
]
}🟢editor_read(action, project_id, include_composition, offset, limit, ...)
scenes: the editor timeline, one scene per segment, as {total, offset, returned, scenes}; scenes build on their own from the storyboard. music: {tracks}, timed in seconds. audio_library: tracks to add with editor_edit; when nothing matches, it lists the library's moods and genres.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "scenes: the timeline (project_id); music: the project's music tracks (project_id); audio_library: music and sound effects to add",
"enum": [
"scenes",
"music",
"audio_library"
],
"type": "string"
},
"project_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Project id"
},
"include_composition": {
"default": false,
"description": "scenes: include each scene's full layer JSON (bulky: page it)",
"type": "boolean"
},
"offset": {
"default": 0,
"description": "scenes: first to return, 0-based",
"type": "integer"
},
"limit": {
"default": 0,
"description": "scenes: how many; 0 returns all",
"type": "integer"
},
"category": {
"anyOf": [
{
"enum": [
"music",
"sfx"
],
"type": "string"
},
{
"type": "null"
}
],
"description": "audio_library: music or sfx (sound effects); omit for both"
},
"search": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "audio_library: free text"
},
"mood": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "audio_library: a mood tag"
},
"genre": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "audio_library: a genre tag"
}
},
"required": [
"action"
]
}🔴editor_edit(action, project_id, segment_number, message, caption_config, ...)
Polish the video in the editor; changes show in the next export. scene_note: edit one scene with a note (move, restyle, add or remove layers and overlays, change the text of a lower third, title card or callout, change motion); spends credits. Returns the changes and skipped_mutations: edits that didn't land (applied=false) or landed differently than asked (applied=true), each with a reason; check them. Scene lengths follow the narration; the image, clip and voiceover layers can't be removed. project_note: a note for the whole video, applied to up to 10 scenes per call; spends credits. captions: merge caption settings. add_sfx: attach a sound effect to a segment. add_music / update_music: background music timed in seconds; a track's duration follows from its trim.
Input Schema
{
"type": "object",
"properties": {
"action": {
"description": "scene_note: segment_number, message; project_note: message; captions: caption_config; add_sfx: segment_number, library_track_id; add_music: library_track_id, optional name, volume, loop, start_seconds; update_music: track_id and any of name, volume, loop, start_seconds, trim_start_seconds, trim_end_seconds",
"enum": [
"scene_note",
"project_note",
"captions",
"add_sfx",
"add_music",
"update_music"
],
"type": "string"
},
"project_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "Project id"
},
"segment_number": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"description": "Segment number (1-based, from storyboard_read)"
},
"message": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "The note in plain words, e.g. \"move the caption to the top\""
},
"caption_config": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "null"
}
],
"description": "captions: keys to merge into the caption settings (get_status shows them)"
},
"library_track_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "A track id from editor_read action=audio_library"
},
"track_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "update_music: a track id from editor_read action=music"
},
"name": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"description": "The music track's name"
},
"volume": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"description": "Music volume 0-1; the 0.12 default sits under the narration, so raise it only if asked"
},
"loop": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"description": "Loop the music under the whole video"
},
"start_seconds": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"description": "Where the music starts in the video, in seconds"
},
"trim_start_seconds": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"description": "update_music: where playback starts within the track, in seconds"
},
"trim_end_seconds": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"description": "update_music: where playback stops within the track, in seconds"
}
},
"required": [
"action"
]
}🔴export_video(project_id)
Render the finished video from the editor timeline; free. When the export job completes, get_status carries export_url.
Input Schema
{
"type": "object",
"properties": {
"project_id": {
"description": "Project id",
"type": "string"
}
},
"required": [
"project_id"
]
}Community
Evidence