What is Reference audio? Definition, key points and related tools
A 3–8-second voice sample bound to a character and sent with each segment so the voice stays consistent. H3 caps total reference audio at 15 s and 3 clips; matching from a preset library is two orders of magnitude cheaper than generating voices per character.
What reference audio fixes, and what it does not
A video model that speaks invents a voice every time it is called. In a series, one character's lines are spread across dozens of segments, so without a fixed sample the same face can sound like different people inside a single episode. A reference clip removes that variable: it carries timbre, pitch and delivery, and the model reuses them whenever that character speaks.
It does not decide the words. Alibaba's documentation for Wan 2.7 reference-to-video states that the voice file serves as a voice reference and does not determine the spoken content (Alibaba Cloud); the line still comes from the prompt. That is also the difference from text-to-speech: TTS delivers a finished line, while reference audio conditions a model that then performs the line together with the picture (see native audio). How preset voices, cloning and native audio compare as routes is covered in the AI voice explainer.
Limits by model
| Model | Audio references per call | Length and format rules | Source |
|---|---|---|---|
| MiniMax H3 | Up to 3 clips | 2–15 s per clip, 15 s in total; WAV or MP3 up to 15 MB; audio input is not charged | API docs, pricing |
| Wan 3.0 | Up to 5 clips | 15 s in total | Alibaba Cloud |
| Wan 2.7 reference-to-video | One voice per reference image or video (up to 5 reference assets) | 1–10 s; WAV or MP3 up to 15 MB | Alibaba Cloud |
| Kling VIDEO 3.0 Omni | Voice taken from a character reference | At least 3 s of speech, or a 3–8 s character video | Kling blog, 2026-07-03 |
| Seedance 2.0 | Up to 3 audio clips (vendor claim) | Not disclosed in the launch post | ByteDance Seed, 2026-02-12 |
| Seedance 2.5 | Up to 10 audio clips (vendor claim) | Not disclosed | ByteDance Seed, 2026-07-31 |
The caps apply per call, not per character. A three-person scene on H3 leaves about 5 seconds of sample for each voice; five speakers on Wan 3.0 leave about 3 seconds each. One mechanical trap on H3: once any reference image, video or audio is in the request, first_frame and last_frame are not allowed, so adding a voice also gives up first-frame continuity for that segment.
Preparing and binding a sample
- One speaker, 3–8 seconds, conversational. Kling asks for references without overlapping voices or loud music. The same logic holds elsewhere: reverb, a music bed or a second voice in the clip become part of what the model copies.
- Match the face first. Gender, apparent age and language come before anything else; a voice that sounds 45 on a character written as 25 is noticed before the story is.
- Audition for delivery, not resemblance. The model is conditioned on the sample's rhythm, so a newsreader or audiobook read pulls every line toward a bulletin. Pick a sample that already rises and falls.
- Name the speaker before every line and keep one label per voice, so the model does not attach the first file to the wrong face in a two-shot.
- Freeze the file. Use the same trimmed clip for the whole series; a different six seconds cut from the same recording can shift pitch and pace between episodes.
Pipeline tools do the binding once in a cast library; SceneMixer, for example, says it matches each character to a voice from a preset library (SceneMixer guide, vendor claim). With a raw model API you store the files and attach them to every request yourself. Either way, the per-call caps in the table above still apply to whichever model renders the segment.
Common mistakes and their causes
- Two voices blended into one. The sample was cut from a dialogue scene, so it contains both speakers. Isolate one.
- An accent that drifts. A sample recorded in the source language pulls pronunciation toward it when the line is in another language. Use a sample in the target language.
- The voice changes when a new character enters. The per-call cap forced one sample out of the request. Plan scenes so everyone who speaks fits inside the cap, or move the extra speaker to the next segment.
- Cloning a recognizable real voice. That is a consent and likeness question before it is a technical one; see the copyright and labeling checklist.
Cost is rarely the constraint. On MiniMax's pay-as-you-go list, audio input is free, Voice Design is $3 per voice, rapid cloning $1.5 per voice, and HD speech $100 per million characters, so synthesizing a 6-second preset sample of about 90 characters costs under one cent (MiniMax pricing). For how many seconds each line should get once the voice is fixed, see dialogue timing.
Key figures and sources
| Figure | Value | Source | Date |
|---|---|---|---|
| MiniMax H3 reference audio | Up to 3 clips; 2–15 s each; 15 s total; WAV/MP3 up to 15 MB; first/last frame not allowed with references | platform.minimax.io | 2026-09-12 |
| MiniMax audio input and voice services | Reference audio input free; Voice Design $3 per voice; rapid voice cloning $1.5 per voice; HD speech $100 per 1M characters | platform.minimax.io | 2026-09-12 |
| Wan 3.0 reference audio | Up to 5 audio clips, 15 s in total per request | www.alibabacloud.com | 2026-09-12 |
| Wan 2.7 reference voice | 1–10 s, WAV/MP3, up to 15 MB; sets the voice, not the spoken content | www.alibabacloud.com | 2026-09-12 |
| Kling VIDEO 3.0 Omni voice reference | At least 3 s of clean speech or a 3–8 s character video; no overlapping voices or loud music | klingai.com | 2026-09-12 |
FAQ
How long should a reference audio clip be?
Three to eight seconds of one person speaking covers what current models ask for: Kling wants at least 3 s of speech, Wan 2.7 accepts 1–10 s, MiniMax H3 accepts 2–15 s per clip within 15 s in total. On capped models longer is not better, because every extra second spent on one voice is a second another speaker in the same call cannot use.
Can I combine reference audio with a first frame?
Not on MiniMax H3: its API rejects first_frame and last_frame whenever a reference image, video or audio is present. Alibaba's Wan 2.7 reference-to-video lists a first-frame image (at most one) among its inputs. Check the model's API reference before planning segment continuity around a first frame.
Does reference audio make a voice identical across episodes?
It fixes the source of the timbre, not the performance: the model re-performs each line, so pitch and pace move with the emotion written in the prompt. Keep the same file, the same trim and the same speaker label for the whole series, and listen across episode boundaries before export.
Sources
- https://platform.minimax.io/docs/api-reference/video-generation-v2-create
- https://platform.minimax.io/docs/guides/pricing-paygo
- https://www.alibabacloud.com/help/en/model-studio/wan3-video-generation-api-reference
- https://www.alibabacloud.com/help/en/model-studio/wan-video-to-video-api-reference
- https://klingai.com/blog/kling-video-3-omni-native-lip-sync-audio-guide
- https://seed.bytedance.com/en/blog/official-launch-of-seedance-2-0
- https://seed.bytedance.com/en/blog/one-take-creation-flexible-referencing-introducing-seedance-2-5
- https://scenemixer.com/guides/wan-3-short-drama/