# Speaker Frames

> Find where each speaker's face is, over time, on every camera of a multicam edit, and get a face-track file that Speaker View's tracked framing reads.

Source: https://nodaro.ai/docs/nodes/video/speaker-frames

The **Speaker Frames** node finds where each speaker's face is, over time, on every camera of an edit. It writes the result to a stored face-track file, with boxes per face on each camera's own clock. The **Face tracks** output carries a summary of that file. Wire it into the **Face tracks** input of [Speaker View](https://nodaro.ai/docs/nodes/video/speaker-view#face-tracking), or click **Add face tracking** there, and each speaker's crop follows their face.

- Found in: Video › Cut & Assemble
- API type: `speaker-frames`

Speaker Frames has no credit price at the moment. Every run is refused before anything is charged, with the message "Speaker Frames is not priced yet". The node, its settings panel and the **Run** button say so. You can place the node, connect it and set it up.

Speaker Frames runs on Nodaro Cloud. On a self-hosted install, [connect the install to Nodaro Cloud](https://nodaro.ai/docs/self-hosting/cloud-connect). A connected install builds small detection copies of the cameras it samples and sends only those, never the original files. The master audio and any camera you untick are not sent at all. At the moment, a run from a connected install is refused with the message "Speaker Frames is not available on a connected install yet".

## When to use it
- You want the framing of a podcast or interview render to follow each speaker's face.
- You have a multicam edit from [Edit Plan](https://nodaro.ai/docs/nodes/video/edit-plan) or [Camera Switch](https://nodaro.ai/docs/nodes/video/camera-switch), and you need to know where each person is on every camera.
- You want face positions for one video, without an edit.

## Quick start
### Add the node

Press Tab on the canvas and choose **Video › Cut & Assemble › Speaker Frames**.

### Connect the edit

Wire the **Switched edit (EDL)** output of Camera Switch into the **EDL** input. Wire the **Transcript (named)** output of Camera Switch into the **Transcript** input.

### Choose the cameras

Open the settings panel. Under **Cameras to track**, untick any camera you do not need tracked, for example a close-up that a render shows full frame.

Workflow: Edit Plan writes the edit, Camera Switch chooses the camera for each cut, and Speaker Frames finds each speaker's face on every camera the edit shows.

- Edit Plan → Camera Switch (EDL)
- Camera Switch → Speaker Frames (EDL)

## Inputs
| Input | Required | What to connect |
| --- | --- | --- |
| **EDL** | One of **EDL** and **Video** | The edit, from [Edit Plan](https://nodaro.ai/docs/nodes/video/edit-plan) or [Camera Switch](https://nodaro.ai/docs/nodes/video/camera-switch). A clip pack runs once: when you wire the **Clips** output of Edit Plan, every clip is folded into one run over all the footage the clips keep, never one run per clip. |
| **Video** | One of **EDL** and **Video** | One video, sampled whole, up to 180 minutes. |
| **Transcript** | No | The **Transcript (named)** output of Camera Switch, with its speaker names. The tracks are then labelled with the names the edit uses. Wire the same transcript that Camera Switch outputs. |

Connect exactly one of **EDL** and **Video**. With both connected, the run is refused before anything is charged.

## Outputs
| Output | What it carries |
| --- | --- |
| **Face tracks** | A summary of the stored face-track file: its address, its SHA-256 hash and size, the sample rate, and for each camera its sampled stretches, scene cuts and tracks. Each track has an id, the speaker when one is known, and its number of boxes. The boxes themselves are in the stored file. |

On a connected self-hosted install, the face-track file is copied into the install's own storage after the run, and its hash is checked again. The output then points at that copy.

## Settings
| Setting | What it does |
| --- | --- |
| **Cameras to track** | Every camera the connected edit samples, with its number of frames. Untick a camera you do not need tracked, and it is not sampled. A camera you unticked that is no longer in the edit is ignored when the node runs. |

The node samples only what the edit keeps: each kept stretch, plus 2 seconds on each side, at 2 frames per second, on every video camera the edit shows. The sample rate is fixed. A camera whose kept footage is longer than 180 minutes is refused before anything is charged.

### Corrections to speaker labels

The node keeps corrections to the speaker of each track, by track id. A correction can also clear a track's label. You set the corrections in **Edit framing…** on [Speaker View](https://nodaro.ai/docs/nodes/video/speaker-view#face-tracking): it draws the tracked faces over each camera, and choosing who a face is saves the correction on this node. The settings panel, the summary on the canvas and Speaker View apply them. A correction keeps the transcript's own label, so after you rename a speaker in Camera Switch, the correction follows the new name. The **Face tracks** output carries the result as the run wrote it, without the corrections. A new run keeps the corrections whose tracks it still finds. The panel lists any correction whose track the new result no longer has, so a correction is never dropped without notice.

## Credits
Speaker Frames has no credit price at the moment. Every run is refused with the message "Speaker Frames is not priced yet", and nothing is charged.

## Tips
- **Run Camera Switch first** on a multicam shoot, and wire its named transcript, so the tracks carry the speaker names the edit uses.
- **Untick the cameras you do not need.** A close-up that a render shows full frame needs no tracking, and fewer cameras mean fewer frames to sample.
- **Cut the edit before you track.** The node samples only the footage the edit keeps, so a tighter edit is sampled faster.

## Troubleshooting
| Message or problem | Cause and fix |
| --- | --- |
| "Speaker Frames is not priced yet" | Speaker Frames has no credit price at the moment, so every run is refused. |
| "Wire an EDL into “EDL”, or a video into “Video”" | Nothing is connected. Connect an edit or a video. |
| The run is refused with both inputs connected. | Connect only one of **EDL** and **Video**. |
| A camera is refused as too long. | The camera's kept footage is longer than 180 minutes. Keep less footage, or untick that camera. |
| The tracks are not labelled with the right names. | Wire the **Transcript (named)** output of Camera Switch, not the transcript of Transcribe. |

## From the API
`POST /v1/speaker-frames` takes `edl` or `videoUrl`, exactly one of them, with optional `transcript` and `excludeSourceIds`. `edl` is one EDL, a clip pack (an array of EDLs) or a clip set, as objects or JSON strings. `excludeSourceIds` lists the cameras you unticked. The finished job's `output_data.json` is the summary of the face-track file.

Requests are refused before anything is charged:

| Error | Cause |
| --- | --- |
| `400 invalid_edl` | The request is malformed, or the edit is not on the recording's clock. |
| `400 invalid_input` | Both or neither of `edl` and `videoUrl` are sent, or `excludeSourceIds` names no camera of the edit. |
| `400 too_long` | A camera is longer than 180 minutes. |
| `503 not_priced` | Speaker Frames has no credit price at the moment. |

On a self-hosted install, the route answers `503 nodaro_connection_required` without a Nodaro Cloud connection, or when Nodaro Cloud rejects the install's connection. Reconnect from **Integrations**. It answers `503 speaker_frames_relay_unavailable` while Nodaro Cloud does not take runs from a connected install, or cannot be reached. If a detection copy the install built is larger than the 500 MB a relay can send, the job fails and names the camera. Keep less footage or untick that camera.

`GET /v1/speaker-frames/capabilities` answers `{ relay, source }`: whether this server runs a Speaker Frames job relayed from a connected self-hosted install. `source` is `server`, `nodaro.ai`, `nodaro.ai-unreachable`, `nodaro.ai-rejected` or `not-connected`. See [Run a single node](https://nodaro.ai/docs/developers/api/nodes) and [Jobs](https://nodaro.ai/docs/developers/api/jobs).

## Frequently asked questions

### What does the Speaker Frames node do?

Speaker Frames finds where each speaker's face is, over time, on every camera of an edit. It stores the boxes in a face-track file, and its Face tracks output carries a summary of that file for Speaker View's tracked framing.

### Can I run Speaker Frames now?

Not yet. Speaker Frames has no credit price at the moment, so every run is refused with the message "Speaker Frames is not priced yet" before anything is charged. You can place and set up the node.

### Does Speaker Frames analyze the whole recording?

No. With an EDL connected, it samples only the footage the edit keeps, plus 2 seconds on each side, at 2 frames per second. A 90-minute episode cut down to 60 minutes is sampled for about 60 minutes.

### Does a clip pack run Speaker Frames once per clip?

No. When you connect the Clips output of Edit Plan, every clip is folded into one run over all the footage the clips keep.

### Which transcript should I connect?

Connect the Transcript (named) output of Camera Switch, the same transcript Camera Switch outputs. The face tracks are then labelled with the speaker names the edit uses.