Pipelines

client.pipelines runs story-to-video pipelines: multi-stage productions that turn a story prompt into a film. A pipeline moves through eight stages, script, characters, objects, locations, shot_list, scene_images, animate_audio_edit and post_merge, and it can stop at each one for your approval. It is the programmatic version of Create film in the studio. The methods call the Pipelines REST API.

Modes

A pipeline's status is queued, running, awaiting_approval, completed, failed, cancelled or forked.

Methods

The scopes apply to OAuth tokens. API tokens and sessions are not limited by scopes.

client.pipelines

create(input)

Starts a pipeline. In auto mode it runs to the end on its own; poll get() for its status and getTimeline() for the result. In manual or guided mode, drive it with pendingApprovals(), approveStage() and approveSubGate().

create(input: PipelineInput): Promise<{ id: string }>

const { id } = await client.pipelines.create({
  story_prompt: "A lighthouse keeper finds a message in a bottle that predicts tomorrow's storm.",
  target_duration_seconds: 60,
  format: "short_film",
  root_node_id: "pipeline-root",
  mode: "auto",
})

get(id)

Reads a pipeline's current state. Poll it to follow an auto run to the end.

get(id: string): Promise<PipelineRecord>

const pipeline = await client.pipelines.get(id)
console.log(pipeline.status, pipeline.current_stage, pipeline.current_progress_message)

A PipelineRecord has id, status, current_stage, mode, spent_credits, reserved_credits, upfront_credit_estimate, failure_reason (set when the status is failed), current_progress_message, and, for a branch, branched_from_pipeline_id and branched_from_stage.

list()

Lists your pipelines, newest first.

list(): Promise<PipelineRecord[]>

const pipelines = await client.pipelines.list()

cancel(id)

Stops a running pipeline. Unspent reserved credits are refunded. Cancelling a pipeline that has already ended changes nothing.

cancel(id: string): Promise<{ ok: true }>

await client.pipelines.cancel(id)

pendingApprovals(id)

Lists the stages waiting for approval, each with its output. The list is empty during a clean auto run, because the engine approves stages itself.

pendingApprovals(id: string): Promise<Array<{ stage_name: PipelineStageName; output: unknown }>>

const approvals = await client.pipelines.pendingApprovals(id)
for (const { stage_name, output } of approvals) console.log(stage_name, output)

approveStage(id, stage, edits?)

Approves a stage so the pipeline moves on. Pass edits, a JSON Patch, to change the stage's output before it is approved.

approveStage(id: string, stage: PipelineStageName, edits?: unknown): Promise<{ ok: true }>

await client.pipelines.approveStage(id, "script")

await client.pipelines.approveStage(id, "script", [
  { op: "replace", path: "/title", value: "The Keeper's Warning" },
])

rejectStage(id, stage, feedback)

Rejects a stage with a note. The engine runs the stage again and takes the note into account.

rejectStage(id: string, stage: PipelineStageName, feedback: string): Promise<{ ok: true }>

await client.pipelines.rejectStage(id, "script", "Make the story darker and more suspenseful")

approveSubGate(id, gate)

Approves a check inside the animate_audio_edit stage, such as dialogue_recheck, so the pipeline continues with the next step.

approveSubGate(id: string, gate: SubGateName): Promise<{ ok: true; gate: SubGateName; resumed_at: string }>

await client.pipelines.approveSubGate(id, "dialogue_recheck")

getStage(id, stage)

Reads one stage's status, output and critic_feedback. Use it to inspect the script or the plan before you approve it.

getStage(id: string, stage: PipelineStageName): Promise<{ status: string; output: unknown; critic_feedback: unknown }>

const { status, output } = await client.pipelines.getStage(id, "script")

getTimeline(id)

Reads the assembled film: the scenes in order with their durations, the audio URLs and the animation progress. Render it yourself, or hand it to an editor.

getTimeline(id: string): Promise<PipelineTimeline>

const timeline = await client.pipelines.getTimeline(id)
for (const scene of timeline.scenes) console.log(scene.compositeUrl, scene.durationSeconds)

A PipelineTimeline has fps, width, height, scenes (each { compositeUrl, durationSeconds }), musicUrl, narrationUrl, and animateProgress with totalShots, shotsDone and percent.

branch(id, input)

Runs a finished pipeline again from one stage, as a new pipeline. The stages before it are copied as approved. The original pipeline stays completed.

branch(id: string, input: { fromStage: PipelineStageName }): Promise<{ pipelineId: string; clonedStages: string[]; clonedEntities: number }>

const { pipelineId } = await client.pipelines.branch(id, { fromStage: "scene_images" })

chatStage(pipelineId, stage, message)

Sends a message to the director in guided mode. The pipeline's mode must be guided, and the stage must be awaiting_approval. The reply may carry a proposed_change you can accept with applyChatProposal(). Chat works on the stages that support it, such as script.

chatStage(pipelineId: string, stage: ChatEnabledStage, message: string): Promise<{
  turnId: string
  role: "assistant"
  content: string
  proposed_change: ProposedChange | null
}>

const { content, proposed_change, turnId } = await client.pipelines.chatStage(
  id,
  "script",
  "Can you make the protagonist's motivation clearer in scene 2?",
)

applyChatProposal(pipelineId, stage, turnId)

Accepts the change the director proposed in an earlier turn. The change is checked, saved as a new version of the stage, and the stage is approved.

applyChatProposal(pipelineId: string, stage: ChatEnabledStage, turnId: string): Promise<
  | { applied: true; attemptId: string; newOutput: unknown }
  | { applied: false; error: { code: string; detail?: unknown } }
>

const result = await client.pipelines.applyChatProposal(id, "script", turnId)
if (result.applied) console.log("Approved:", result.newOutput)
else console.log("Not applied:", result.error.code)

When the change cannot be applied but you can recover, the result is applied: false and the director has already added a reply with a hint. A hard failure throws a 409 error.

getStageChat(pipelineId, stage)

Reads a stage's chat history. It is empty before the first message.

getStageChat(pipelineId: string, stage: ChatEnabledStage): Promise<{ turns: ChatTurn[] }>

const { turns } = await client.pipelines.getStageChat(id, "script")

Each ChatTurn has id, turn_n, role, content, proposed_change, applied_to_attempt_id and created_at.