W01 — A briefing dubbed into another language, one voice per speaker

A space agency publishes its briefings in English. The same six minutes — the Administrator on the programme, three astronauts on what it means to them — should reach an Italian audience as a video they can simply watch: Italian voices, the right voice for each person, speaking when they speak, over the same room. This example does it with the dubbing engine of Dubbing with natural speech, and it is the same workflow for any other language: change one input.

The figure tells the workflow as stages, with the reference run’s results on the cards; the exact graphs are the short workflow that you run and the engine it calls (as a diagram).

The case

The source is NASA’s “Artemis Media Resource — Interview Soundbites”: press material, a series of answers meant to be cut apart, with a title card before each speaker and short black frames between answers. It is a demanding test for a dub, for three reasons: four people with very different voices and amounts of speech (the Administrator speaks almost two minutes, one astronaut eleven seconds), a lot of technical vocabulary (Artemis, Orion, Gateway, the Space Launch System), and the spontaneous speech of interviews — hesitations, self-corrections, sentences cut across two captions.

Input	Who provides it	Reference run
Source Video	the communications team	the interviews, 6 min 13 s, 960×540
Target Language	whoever orders the version	it
Preserve Background	the editor	true
Generate Subtitles	the editor	true

What it shows

• One workflow, any language. The language is an input, a BCP 47 tag, used by the translation, the adaptation and the voices. The same engine dubbed a three-speaker interview into Spanish, Italian, French, German and Brazilian Portuguese without a change.
• A voice per person, chosen by listening. The transcription tells the four speakers apart in one pass; eight seconds of each are cut from the video and a video model says whether the voice is feminine or masculine; each speaker gets a voice of that kind, their own for the whole video. See One voice per speaker.
• Fit by rewording, measured. Every Italian line is voiced and measured against the time the English line took; a line that is too long is reworded shorter with the measured length in hand, up to three rounds. Seventy-one lines of ninety-six fit at the first attempt.
• Deliver first, and say where to look. The dub is delivered with a quality report that names, with their time, the few lines that needed a faster voice or more time than their window — the places a person should listen to.
• An engine behind a short workflow. The workflow you run has four inputs, one Sub-workflow step and nine outputs; the engine — about eighty nodes — is a separate workflow it calls by its wf_ ID. You can open, study and change the engine without touching what users run.

The workflow, step by step

The short workflow passes its four inputs to the engine and returns its outputs. The engine’s nodes are labelled by phase; the table groups them as Dubbing with natural speech explains them.

Phase	Nodes (label prefix)	What they do here
Listen	A1 Inspect Master, A2 Extract Speech Audio, A3 Keep One Pass up to an Hour, A4 Transcribe with Speakers, A5 Rebuild Source Captions, A6 Speech-Ready Preflight	measure the video (6:13); transcribe it in one pass with Scribe V2, word timing and speakers; 96 lines; the preflight passes
Voices	V1 List Speakers, V2 Cut Each Speaker’s Sample, V3 Hear Each Speaker’s Kind of Voice, V4 Collect Kinds of Voice, V5 Assign One Voice per Speaker	four speakers, an 8-second sample each, one Video Analysis per sample, one voice per speaker
Translate	B1 Build Translation Blocks, B2 Initial Translation, B3 Detect Missing Translations, B4–B5 the repair round, B6 Finalize Localized Captions	96 lines translated in blocks of 24; nothing missing, so the repair round is skipped
Fit the speech	C0 Plan Speech Timing, then per round C1.1–C1.8, C2.1–C2.8, C3.1–C3.8	the windows; three rounds of adapt → voice → measure → reconcile on the open lines only
Dialogue	D1 Enumerate Delivery Cues, D2 Fit Each Cue to Its Window, D3 Collect Cue Evidence, D4 Compose Dialogue Timeline	96 takes fitted and placed on a 6:13 timeline
Background	E1 Extract Original Mix, E2 Gate Optional Background Recovery, E3–E8 segments, Demucs, drums + bass + other, joined	the music and the room, without the English voices, in five 76-second segments
Finish	F1 Mix Dialogue + Background, F2 Normalize Delivery Audio, F3 Measure Final Audio, F4 Replace Video Audio, F5 Inspect Final Video	ducked mix at −16 LUFS; new soundtrack on the untouched picture; length checked
Account	G1 Build Dubbing Manifest, G2–G4 quality gates (warn)	the manifest and the quality report
Subtitles	H1–H4	the original with English subtitles, the dub with Italian subtitles, both SRT files

The reference run

The briefing dubbed into Italian, 6 min 13 s; the original for comparison. The subtitles of both languages are the SRT files the run delivered: Italian, English.

The voices. The four speakers were told apart and cast as follows — speaker numbers are in order of first appearance, voices in order of speaking time (speaker voices):

Speaker	Who	Speaks	Lines	Heard as	Voice
speaker_0	Jim Bridenstine, NASA Administrator	1 min 55 s	44	masculine	Chris
speaker_1	Reid Wiseman, astronaut	1 min 23 s	37	masculine	George
speaker_2	Jeanette Epps, astronaut	11 s	5	feminine	Sarah
speaker_3	Doug Wheelock, astronaut	28 s	10	masculine	Brian

The rounds. 96 lines. Round 1 voiced all of them: 71 fit at natural speed. Round 2 reworded and voiced the 25 still open: 11 fit. Round 3 took the last 14: 13 were accepted, and one, which no rewording fitted, was reflowed into the pause after it. Five lines in all play faster than 1.2×, three of them above 1.5× — and the report names them.

The quality report. passed, all eight checks true, with the four moments a person should listen to (quality report):

SPEECH_TIMING_REFLOWED       [00:02:03.780-00:02:04.190] Line reflowed after the last round: it runs 0.2 s past its line,
                             played at 1.10x; the next 3 line(s) start up to 0.1 s late.
SPEECH_TIMING_DEGRADED_RATE  [00:03:15.360-00:03:16.100] Combined speech rate 1.845 exceeds the 1.50 degraded-delivery threshold.
SPEECH_TIMING_DEGRADED_RATE  [00:04:18.200-00:04:20.740] Combined speech rate 1.887 exceeds the 1.50 degraded-delivery threshold.
SPEECH_TIMING_DEGRADED_RATE  [00:04:45.300-00:04:48.339] Combined speech rate 1.529 exceeds the 1.50 degraded-delivery threshold.

Each is a real limit of dubbing spontaneous speech. At 2:03 the Administrator says a single word, “fuel.”, in four tenths of a second between two sentences: “carburante” cannot be said that fast, so after three rounds the line keeps the word as he said it, in English, and takes a fifth of a second from the pause that follows. At 3:15 “of college.” has three quarters of a second and becomes “usciti dall’università.”; at 4:18 and 4:45 Wiseman’s long, self-correcting sentences (“you’re definitely gonna moto- motivate the”) are kept whole in Italian, faster than he spoke them. A person listens to these four moments and decides whether to accept them — Limits worth knowing explains why such lines happen.

The cost. 41.2 credits for 6 min 13 s — about 6.6 credits per minute:

Step	Credits
Voices — Text to Speech, 96 lines and 39 retakes	19.3
Background — Stem Separation, 6 min 13 s	16.4
Transcription — Speech to Text, 7 started minutes	3.5
Translation and adaptation — Caption Translation	1.8
Casting — Video Analysis, 4 samples	0.2

Without the background (preserve_background: false) the same dub costs about 25 credits, 4 per minute. The run took twelve minutes: the rounds wait for each voice take, and the separation runs meanwhile.

Run it yourself

node madoo-install-example.mjs w01-dubbed-briefing --api <your Madoo API URL> --key <key.json> --run --save dubbed

The installer uploads the video, creates and publishes the engine, then the short workflow that calls it, and runs it; --save downloads the outputs. Installing an example explains the installer.

To dub your own video, run the installed workflow with your video and language. To change what it does:

You want	Change
another language	the Target Language input: es, fr, de, pt-BR, ja…
the dialogue only, no background	Preserve Background false: no separation, about 40% cheaper
no subtitles	Generate Subtitles false: both subtitle branches are skipped
your own cast	in the engine, V5 Speaker Voices: fixed voices by speaker on Voices, or your own voice lists
a brand’s vocabulary kept	in the engine, the same Glossary on every translation step: B2, B5 and the three adaptations C1.2, C2.2, C3.2

Media: “Artemis Media Resource — Interview Soundbites”, NASA (public domain; NASA does not endorse this documentation). The title card with press contacts at the start of the original is not part of the example.