Dubbing: quality, costs and limits

A dub is delivered with the evidence to judge it: a quality report with the verdict and the moments to listen to, a manifest with every line, take and measurement. This page explains how to read them, what a dub costs and how long it takes, and where its limits are — the things to know before promising a dub to someone, and the things to check after a run. Dubbing with natural speech explains how the engine works; Building a dub from its blocks how it is wired.

Checking a dub

Review a dub in this order; each step tells you whether the next is needed.

1. Watch the dubbed video at normal speed, with the sound on. A dub that sounds right to a listener is right.
2. Read the quality report (quality_report). passed: true with no warnings means every line was voiced, fitted and placed, the soundtrack and the video are the master’s length, the loudness is on target. Each warning carries a time range — [00:02:03.780-00:02:04.080] — and says what happened there.
3. Listen to the moments the warnings name. They are the few lines that play faster than natural or that took time from what follows. Decide whether each is acceptable; most are.
4. Open the manifest (dubbing_manifest) only to understand a line: its source and dubbed text, its window, its speaker and voice, the length of its take before and after fitting, and the rate it was played at.
5. In the engine run, two diagnostic outputs go further: speech_timing_history, every take of every round with its text, length and verdict, and speech_timing_quality, the final decision line by line. The dialogue timeline (dialogue_timeline) is the dub without the background: listen to it to tell a voice problem from a mix problem.

What the numbers mean

The rate of a line is how much faster than its natural pace a take is played to fit its window. Up to 1.10× no listener hears it; up to 1.20× it sounds brisk; above 1.50× it sounds hurried. The report counts the lines above 1.20× and names those above 1.50× (SPEECH_TIMING_DEGRADED_RATE); a line that no take could fit even at 2.0× is reflowed into the time after it and named too (SPEECH_TIMING_REFLOWED), with how much later the following lines start.

Results vary from run to run. The translation, the rewording and the voice are generated: the same video dubbed twice gives two dubs, equally correct, with different wording and different quick lines. The six minutes of W01, dubbed into Italian three times, had one, five and three lines above 1.5× — lines of one to three seconds where the Italian needs more syllables than the English. The quality report is what tells the two runs apart, and it is why every warning comes with its time.

What reduces quick lines:

• More room. A line may borrow the pause after it: raise the plan’s Maximum End Extension from 1000 up to 2000 ms. It helps lines followed by a pause, not lines followed at once by the next.
• Language. German and French need more syllables than Spanish and Portuguese: expect a few more quick lines, and a few more takes.
• The source. Spontaneous speech — a sentence cut across two lines, a correction, a one-word line — is the hardest to fit. A scripted voice-over dubs almost without warnings.
• A person’s decision. For a video that must be perfect, a reviewer listens to the named moments and either accepts them or edits the lines; a Quality Gate in route mode can hold the dub back until then.

What it costs

Four kinds of step are paid; measuring, fitting, mixing, loudness, subtitles, the manifest and the checks cost nothing. Measured on real runs, per minute of video:

Step	Credits per minute	What it depends on
Voices — Text to Speech	2.7 – 3.6	the characters spoken, rounds included: the language and how much is said
Background — Stem Separation	2.6	the length of the video; nothing when the background is off
Transcription — Speech to Text	0.5 – 0.6	the length of the video, by the started minute
Translation and adaptation — Caption Translation	0.25 – 0.3	the text; the rounds’ rewording is short
Casting — Video Analysis	about 0.05 per speaker	the number of speakers

By language, on the three-and-a-half-minute interview of the translation lab:

Target	With background	Without background
Spanish (es)	6.2 per minute	3.5 per minute
Brazilian Portuguese (pt-BR)	6.2	3.6
Italian (it)	6.4	3.8
French (fr)	6.8	4.2
German (de)	7.1	4.5

The six minutes of W01 in Italian cost 41.2 credits, 6.6 per minute. A video with long silences costs less than one of continuous talk: the voices are billed on what is said, not on the length of the video.

Estimate before, settle after. estimate_execution gives the expected cost before a run. How many takes the rounds will need, and how long each voice will speak, is known only as the run measures it: each step then reports what it actually consumed, and the run’s credits_used is the real total.

Time. A dub takes longer than the video: twelve to fifteen minutes for the six minutes of W01. The rounds wait for each voice take, the separation runs meanwhile, and the takes of a round are generated side by side. Plan for runs, not for live use.

Limits worth knowing

Speaker identity holds within one pass. One transcription names its speakers consistently from start to end, and the engine keeps a recording of up to about an hour in one pass. A longer recording is transcribed in segments, and speaker numbers restart in each: a voice may change at a segment border. The manifest says so (SPEAKER_LABELS_SEGMENT_SCOPED).

One-word lines between two sentences. A word said in a few tenths of a second — “fuel.” at 2:03 in W01 — has no room for its translation: “carburante” cannot be said that fast. After the rounds, such a line is either played fast or, as in W01, kept as the speaker said it, in English, and reflowed into the pause after it. The report names it. It is a property of spontaneous speech, not of a language; a scripted text rarely has it.

Overlapping speech is voiced in turn. When two people speak at once, the transcript keeps two lines, one after the other; the dub voices them in turn.

Lips are not moved. The dub fits each line to when the speaker speaks, not to the shape of their mouth. On a wide shot nobody notices; on a close-up a careful viewer sees that the words differ.

The background is as clean as the separation. The separation model removes the original voices well from music and ambience, less well from a noisy room: a trace of the original voice may remain under the dub.

The kind of voice is inferred. A video model listens to eight seconds of each speaker; it placed all four speakers of W01 correctly, but it is an inference. For a known cast — a presenter, a brand voice, a public figure — fix the voices by speaker.

Voices are stock voices. Each speaker gets a voice of their kind from the voice model’s library, not their own voice.

Every step depends on its provider. A voice, translation or analysis provider that refuses a request — a quota exceeded, a service unavailable — is retried; if it keeps refusing, those iterations fail. The run then ends partial_success, and its error_message names the step, how many of its iterations failed and why, down to a step inside the engine: “2 of 4 iterations of step “V3 · Hear Each Speaker’s Kind of Voice” failed (RATE_LIMIT) …”. When the failing step is inside the engine, the ready workflow does not deliver the outputs of that run; run it again when the provider is available.

Warnings and errors

Code	Where	Meaning	What to do
SPEECH_TIMING_DEGRADED_RATE	quality report	a line plays above 1.5× after three rounds	listen at the time given; accept, or give the line more room
SPEECH_TIMING_REFLOWED, SPEECH_TIMING_DEGRADED_REFLOW	quality report	a line took time from what follows; the following lines start up to the given delay late	listen from that line on
SPEAKER_LABELS_SEGMENT_SCOPED	manifest	the recording was transcribed in segments; speakers restart in each	expect voice changes at segment borders
SPEECH_TIMING_NO_AUDIBLE_SIGNAL	rounds’ history	a take came back silent; it is voiced again next round	nothing, unless it persists
TRANSCRIPT_SPEECH_EMPTY	Aggregate Transcript, run failed	the video has no usable speech; nothing was paid after the transcription	check the video has speech in the audio track
NATURAL_SPEECH_WIRING	validation	a dubbing build is wired wrongly; the message names the node and the port	see the wiring rules
RATE_LIMIT, SERVICE_UNAVAILABLE	a step, run partial_success	a provider refused requests even after retries	run again later
CHILD_PARTIAL_SUCCESS	the Sub-workflow step	the engine completed partially; the message carries its reason	read the reason; run again
SPEECH_TIMING_PROVENANCE_MISMATCH	the final reconcile or the manifest	a take or a state does not belong to this run’s plan	rebuild the wiring from the engine: states and takes must come from the same run
SPEAKER_VOICES_INVALID	Speaker Voices	a voice list is empty	give each list at least one voice