Speaker View

The Speaker View node renders an edit with its speakers on screen. It can place two hosts side by side, follow whoever is speaking in a vertical clip, show a panel in a grid or use a picture-in-picture, with a crop framed on each speaker. It switches at each speaker change and puts an emphasis on whoever is talking, and the sound is the edit's own, passed through.

Speaker View is priced per minute of rendered output. The rate depends on the Quality and on how many speakers the render puts on screen at once. See Credits.

Speaker View runs on Nodaro Cloud. On a self-hosted install, connect the install to Nodaro Cloud, and the node relays each run to the cloud. A self-hosted install sends each camera file to Nodaro Cloud, so a private camera file over 500 MB is refused before the run. Use a public URL or a smaller file.

When to use it

• Show both hosts of a podcast side by side, with an emphasis on whoever is talking.
• Make a vertical clip that follows the speaker, with a crop framed on each person.
• Show a panel of up to six speakers in a grid.
• Keep one speaker large and the other in a small inset, with a picture-in-picture.

Quick start

Speaker View decides how the speakers are shown, never who is on screen. Camera Switch chooses the camera for each cut, and Speaker View lays the cameras out. A typical chain is Audio Sync, then Transcribe with speaker labels, Edit Plan, Camera Switch and Speaker View.

Inputs

When the node wired into EDL has not run yet, the settings panel says so and offers Run up to here. It runs the nodes before Speaker View, but not Speaker View itself.

Outputs

Each take keeps its own EDL and Transcript. When you pick an older take in the results, its own EDL and Transcript come back.

Settings

The settings panel greys out a choice that the aspect ratio or the number of speakers rules out, and says why when you hover it or move the keyboard focus onto it. The reasons are also listed under the group, for touch screens.

Which choices apply where

• Side by side is drawn for 16:9 and 1:1, and Stacked for 9:16, 4:5 and 1:1. Grid, Single and Picture in picture are drawn for every aspect ratio.
• Side by side, Stacked and Picture in picture take exactly two speakers. Grid takes two to six. The slots are the speakers in the order they first appear.
• A layout that the aspect ratio or the number of speakers rules out renders as its twin (Side by side and Stacked), or else as Grid if the number of speakers fits, or else as Single.
• Pan sweeps inside one camera. At a speaker change between two cameras, it cuts. Under the Switch choices, the panel counts how many speaker changes each switch reaches.
• Crossfade is used only where the edit jumps in time, because a stretch of the recording was cut out between the two speakers. It is never used over continuous speech.

Framing

Edit framing… opens the framing editor over the whole window. It needs the edit's cameras and speakers, so run Edit Plan, and Camera Switch, first. When no camera has a speaker to frame yet, because no transcript with speaker labels and no Edit Plan names them, the editor says so instead of showing a frame with nothing to draw. Connect Transcribe or an Edit Plan so that Speaker View knows who is talking. Until then, Reset to thirds and Save framing stay off. The editor does not open on a workflow that is shared with you for viewing only.

• Cameras and speakers. The editor lists each camera of the edit, and under it the speakers it shows. A camera that shows one speaker, such as a close-up, is shown full frame until you click Draw a box. A camera shared by several speakers, such as a wide shot, starts with one box per speaker. Reset to thirds brings those boxes back.
• The frame. The camera's own video plays under the boxes, with a scrubber in the edit's master time. Jump to a speaker seeks to their first turn longer than 3 seconds on that camera.
• The crop. A box is a free rectangle. The renderer crops it to the shape of the layout's slot around its centre, so one box serves every layout and aspect ratio. Choose a layout under Show as to see its crop as a dashed rectangle. A label states the size of the crop and how much it is enlarged, and it turns amber above 1.5×, where the picture starts to soften.
• Keys. The arrow keys move the box by one camera pixel, or ten with Shift. Tab moves through the boxes, and F shows the selected speaker full frame. Swap trades the boxes of a camera's two speakers.
• Saving. Save framing stores every box shown. Nothing is stored until you save, and closing with unsaved changes asks first. A node whose framing was never saved renders every camera full frame.
• Faces. With face tracks wired, the editor draws the faces that Speaker Frames tracked as dotted boxes, labelled with the track and its speaker. See Face tracking.

Split the camera by speaker frames a wide shot without drawing anything. With the switch on and no framing saved on the node, each speaker on the one camera gets the default third: full height, the output's shape, centred on that speaker's share of the frame. Speakers are dealt the thirds in the order they first appear.

• When it applies. One edit with one camera and 2 to 6 speakers on screen, at an output other than 16:9. One speaker, more than six, several cameras or a run over several clips stay full frame. A 16:9 output stays full frame too, because each box would be the whole frame.
• What wins. The boxes assume a 16:9 camera. Framing you saved always wins. With face tracks wired, a speaker that a track names follows their face, and the default thirds frame only the other speakers.

Face tracking

With face tracking, each speaker's crop follows their face instead of a fixed box. Wire the Face tracks output of Speaker Frames into the Face tracks input. Or click Add face tracking in the settings panel. It adds a Speaker Frames node that reads the same edit and the same transcript as Speaker View, and connects its tracks, as one undo step.

Speaker Frames has no credit price at the moment, so it cannot run yet. A Speaker View with face tracks wired is refused until Speaker Frames has a result.

• Same transcript. Speaker Frames must read the same transcript as Speaker View, the Transcript (named) output of Camera Switch. The tracks then carry the speaker names the edit uses.
• What follows a face. Each slot that shows a speaker on a camera follows that speaker's face track. Where the track has a gap of more than 2 seconds, the crop is placed from where the face is elsewhere in the clip, or else from your Edit framing… crop. A camera you unticked in Speaker Frames always uses your framing.
• Motion. Glide moves the crop smoothly with the face and holds still while the face stays near the middle. Static frames each segment once from the face. With either choice, the size of the crop does not change during a speaker's turn.
• Who a track is. A track takes the speaker name that Speaker Frames gave it, renamed through Camera Switch. A correction on the Speaker Frames node comes next. A track that is still unnamed takes the name of a box you drew in Edit framing…, when the middle of the face is inside exactly one box. A box the editor drew for you never names a track.
• Correct a track. In Edit framing…, click the label of a face, or use the list under the frame. Choose a speaker, Nobody, or As detected to remove a correction. Save framing stores the corrections on the Speaker Frames node. Nothing runs again.
• Coverage. The badge notes a speaker whose time on a camera is less than 60% covered by tracks. The render uses your framing for the rest of that time.
• Render final and Update preview use the saved tracks. They never run Speaker Frames again, so the preview and the final use the same tracks.

Credits

Speaker View is priced per minute of rendered output, rounded up to whole minutes, with a minimum of 1 minute. The rate depends on the Quality of the render and on its tier. A preview has a lower rate than a final. The node shows the exact price before it runs.

• One tier for the whole render. A single segment with three speakers on screen puts the whole render at the higher rate.
• The tier follows the layout as drawn. Side by side, Stacked or Picture in picture over three or more speakers is drawn as a Grid, so it uses the higher rate. A Zoom switch under Side by side or Grid draws nothing and adds nothing.
• Pan, Cut, the crossfades and the emphasis do not change the tier.
• Reserved on the edit, charged on the render. Credits are reserved from the length of the edit when the render starts. Crossfades can make the finished cut shorter. You are charged for the minutes of the rendered cut, never more than was reserved, and the rest is released.
• Previews and finals are separate renders. Each one is charged at its own rate. A clip pack renders and charges each clip on its own, at that clip's length and tier.

Before a run, the node, the Run button and the Render final and Update preview buttons show an estimate. When the number of speakers is not known yet, only Single without the Zoom switch is quoted at the lower rate. Every other layout is quoted at the higher rate, so the estimate can be higher than the charge, but never lower.

Tips

• Run Camera Switch first on a multicam shoot. Speaker View never picks a camera, and it refuses a multicam edit that names no speaker.
• Follow the overlaps. Turn on Layout hints in Camera Switch and set the Layout of Speaker View to Auto, to show a side-by-side layout when two people talk at once. Apply EDL refuses an edit with layout hints.
• Switch back to Apply EDL. Back to Apply EDL in the settings panel replaces this node with an Apply EDL node in place, as one undo step. Apply EDL offers the reverse, Replace with Speaker View.
• Choose the aspect ratio first. It decides which layouts are offered.

Troubleshooting

From the API

POST /v1/speaker-view takes edl, one EDL as an object or a JSON string, with optional transcript, quality, targetAspect, layout, switch, emphasis, accentColor, speakerRegions, tracks and motion. layout is one of auto, single, side-by-side, stacked, grid or pip. switch.type is cut, pan, zoom or xfade:<transition>, and emphasis.style is a +-joined set of scale, border and dim, or none.

tracks is the Face tracks output of Speaker Frames, and motion is static or glide. motion defaults to glide and is ignored without tracks. A refused request answers 400 with invalid_edl, unsupported_setting, too_long or invalid_tracks. The render reserves and charges as described in Credits.

The finished job's output_data has videoUrl, thumbnailUrl, json (the EDL as drawn) and transcript (the wired transcript remapped through that EDL). See Run a single node and Jobs.