What does this solve?
Voice is needed where the text changes every day: order status in a telephone robot, notifications in the application, a training video that is re-released after each amendment of the regulations, an audio version of the newsletter. An announcer for each edit is a rate per shift, studio and two days of approvals. Synthesis removes this dependence and provides voice acting on demand in seconds. In return, you have to keep a dictionary of accents, monitor the normalization of numbers, and put up with even intonation in long texts.
How it works
Conveyor
- Text normalization. Numbers, dates, amounts, units, abbreviations, abbreviations, Latin alphabet inside a Russian phrase are turned into words. “12,500 ₽”, “t. e.”, “OOO”, “USB-C” - each case is decided by a rule. An error at this step is heard instantly and sounds like a defect.
- Accents and homographs. Russian requires a separate step: “castle”, “road”, “stands” are read in two ways without context. Many engines accept the accent mark directly in the text, and this is the main control lever.
- Conversion to phonemes. Some systems have an obvious step with a phoneme dictionary, while in end-to-end neural network systems it is hidden inside the model.
- Acoustic model. From phonemes and prosody, a spectrogram is obtained: what sounds, for how long, with what intonation.
- Vocoder. The spectrogram turns into a sound wave.
- Coding. PCM or wav for telephony and editing, mp3 or opus for the network.
Engine classes
| Class | What gives | What do they pay? |
|---|---|---|
| Cloud APIs | the most natural sounding, many voices and languages, markup support, no need to expand anything | text goes out, payment for volume, network delay, dependence on availability |
| Local fast (Piper, Silero, RHVoice) | work on the processor faster than real time, offline, without paying per character | voices are simpler, intonation is smoother, there are fewer languages |
| Cloning and expressive (XTTS, F5-TTS and relatives) | short sample voice, more emotions | is heavier in terms of iron, less stable from run to run, legal risks |
Pronunciation control
SSML markup specifies pauses, speed, spelling, number reading, and sometimes substitution of entire word pronunciation. Support for engines varies, and it is checked against the specific documentation, and not against the general idea of the standard. In addition to markup, voice selection, tempo, pitch, format and sampling frequency can be adjusted.
Latency and Throughput
For dialogue and telephony, the decisive metric is the time to the first sound: the synthesis gives the first piece without waiting for the end of the phrase, and the robot begins to speak while it counts the rest. For batch audio, the real-time coefficient is more important: how many processing seconds are spent per second of sound.
The telephone channel operates at 8 kHz with a narrowband codec. A voice that is luxurious in headphones may sound worse in a handset than a simple voice specially prepared for telephony. You need to listen to candidates in the tract where they will work.
Cache and pre-chopped phrases
In most product scripts, the text consists of constant chunks and a small variable part. “Your order number”, “will be delivered”, “thank you for contacting” - this is static information that is synthesized once and stored in files. The variable part - name, amount, date, address - is synthesized on the fly or assembled from a pre-announced set of numbers and months.
This assembly solves three problems at once: it removes the delay on constant phrases, cuts the count for symbols in the cloud, and gives a stable sound that does not change from run to run. The cache key is built using the pair “text plus voice parameters”, otherwise after changing the tempo in the settings, the system will continue to serve old files. The junction of static and dynamics is listened to separately: different volumes and different lengths of pauses on the splice are heard better than any defect within the phrase.
Voice cloning
Technically you need a few seconds to a few minutes of clean recording. Legally, written consent of the voice speaker with a list of use cases is required. The voice of an employee who has quit cannot continue to be used without separate permission, and this is not a matter of taste, but of claim.
Useful scripts
- Telephone robot: static phrases are synthesized in advance and stored in files, the variable part (name, amount, date) is synthesized on the fly with low latency priority.
- Voice-over of training and product videos, the text of which is edited weekly and will not survive re-recording by the announcer.
- Audio version of articles and newsletters for those who listen while driving and on the road.
- Local offline notifications on a device, kiosk or terminal where there is and will not be Internet.
- Rough voiceover for editing: timing and rhythm are checked before the studio is paid.
- Localization of the finished video: translation of the track into a second language without reshooting and without a second announcer on staff.
Limitations
- Accents and homographs in Russian are the main source of shame. Names, toponyms and industry terms require a dictionary of exceptions, which is manually maintained and updated based on complaints.
- Numbers, units, contract numbers and articles are read unpredictably. It’s more reliable to expand them into text on your side than to rely on the engine’s normalizer.
- Emotions are limited even in the best models, and in long text the intonation floats between pieces. A smooth result is obtained by cutting into phrases and stitching with pause control.
- Cloud synthesis means delaying and transmitting the text outward, and the text often contains the name and amount of the order.
- Paying by symbols forces you to count in advance: monthly volume, share of repeated phrases, cache of ready-made files. Without a cache, the account for standard notifications grows in vain.
- Open model and voice licenses vary, and some do not allow commercial use.
- The live announcer wins where delivery and play are needed. Synthesis is good on utilitarian text and on volumes that cannot be covered by humans.
- The sound of the voice is not preserved: updating the engine version can change the timbre and tempo, and with them, brand recognition.
How to check the result
- Collect an “evil list” from your own texts: customer names, addresses, amounts, abbreviations, numbers, product names. Run the same list through all candidates.
- Arrange a blind comparison: identical files without signatures, three or four people, assessing naturalness and legibility separately.
- Measure the time to the first sound and the real time coefficient on the target hardware and at the target text size.
- Listen to the result in a real path: telephone line, cheap speaker, headphones, noisy car.
- Run a long text as a whole, and not just one indicative phrase: defects in intonation and breathing appear in the third minute.
- Calculate the cost of the monthly volume of symbols and separately calculate what proportion of phrases are static and go into the cache.
- Fix the version of the model and voice, and after each update, repeat the run of the same “evil list” and compare with the saved files.
Rule: the engine is selected according to three dimensions on its own material - delay to the first sound, cost of a monthly volume and blind listening in the target tract.