Custom.MT
  • Home
    • For Localization Teams
    • For LSP
    • For Product Managers
    • For Translators
  • Services
    • Machine Translation Model Fine-Tuning
    • Machine Translation Evaluation
    • On-Premise Machine Translation
    • Translation Memory (TMX) Cleaning
    • Language dataset acquisition
    • Workshops – Train Your Team in Language AI
  • Products
    • AI Translation Platform
    • Custom Translation Portals
    • For Trados
    • For Smartling
    • For memoQ
    • Shopware Translation Plugin
    • API
    • Documentation
  • Resources
    • Blog
    • Case Studies
    • Events and Webinars
      • GenAI in Localization
    • MT Leaders
  • About Us
    • About Us
    • Terms and Conditions
    • Privacy Policy
  • Book a Call
  • Sign in

Search

The Arrival of Automatic Dubbing
  • Blog post
  • Related Posts

The year 2021 marked the arrival of speech to speech translation in the commercial world.

Scientists are working on making the underlying technology smoother and more accurate, engineers are integrating it into practical use cases. At the same time, there is an explosion in neural voices. Between July and September, three companies in this area got funded, Murf.ai, Wellsaid Labs, and Lovo.ai. As a result, localization professionals can now choose from 20+ providers of pre-made voices, or even clone their own voice to generate personalized speeches.

Speech Translation Full Steam Ahead

Lately, big Tech providers and other important players introduced some major changes in their speech-to-speech translation technologies. Among others, Amazon AI showcased some of their work at MT Summit conference in August. In the presentation conducted by Marcello Federico, prosodic alignment was an important aspect of the work. Google introduced “Translatotron-2” in August, which outperforms the original model. Iterations resulted in better translation quality and more natural predicted speech. The new model also enables the retention of source voice in translated speech. In Academia environment, ELITR project run by Ondrej Boyar from the Charles University in Prague and funded by Horizon Europe 2020 arrives at its conclusion in December.

Neural Voices into Products

In a live voiceover scenario, the source can come from MT. In an “automated dubbing” scenario, an engineer can edit the text and the pronunciation of the neural voice to improve on quality. As technology comes to the market, this type of offering has begun to crystallize into “AI voiceovers” at an average price of $20 per minute.

The screenshot displays interface of Resemle.AI - an AI voice generation software.

Yandex Youtube Translator – First Impressions

In September, Russian’s Tech  Giant Yandex unveiled a free automatic English- Russian translator for Youtube (as well as other video streaming services). Through a “Translate” button added to Yandex Browser (Chrome compatible), it translates videos and dubs them with the recognizable cheer of Alisa, Yandex’s sister system to Amazon Alexa. Under the hood, the translator chains several neural networks to make it sound more natural.

At Custom.MT, we took the translator for a spin!

Overall, the feature left a favorable impression: we could follow TED talks and narrated videos with clear diction in Russian. Even though the translation was literal, it was clear enough to follow the speaker. In a more specialized video with hardcore terminology, the MT engine predictably stumbled. Here is what our team noticed:

  • Youtube translator works better for short videos. A 30-minute clip took 8 minutes to render.
  • It understands the need for speed. Russian sentences may get longer than English. To compensate, the digital narrators change their delivery. They sometimes spray words at a comic rate to keep up with rapid-fire jabber.
  • Prosody is in! The voice changed in pitch depending on the source pitch and tone. It felt close to a human voice on shorter stretches.

David Talbot, the head of NLP at Yandex, told us they’re planning on adding more translation pairs and voices to the Yandex Youtube translator, and also on making it faster in the upcoming future.

Considering billions of videos on Youtube to potentially translate, the processing power needed to run so many neural networks behind the service, and it being completely free, this is an ambitious project!

A frame from a video of David Talbot talking about Yandex video translation feature

dubbingmachine translationspeech-to-speech
Kate Vostokova
Kate Vostokova

Comments are closed.

Stay in the loop
Subscribe to receive the latest industry news, updates on MT & LLM events, and product information

Categories

  • Blog post
  • Case studies
  • Guides
  • Infographics
  • Interview
  • Press Release
  • Related Posts
  • Uncategorized
  • Webinars

Webinars

  • AI Prompt Engineering for Localization – 2024 Techniques
  • AI Prompt Engineering for Localization
  • Managing Machine translation in LSPs in 2023
  • Natural Language Processing for Business Localization (Webinar)
  • Let’s Machine Translate Our Website!
  • hello@custom.mt
  • Home
    • For Localization Teams
    • For LSP
    • For Product Managers
    • For Translators
  • Services
    • Machine Translation Model Fine-Tuning
    • Machine Translation Evaluation
    • On-Premise Machine Translation
    • Translation Memory (TMX) Cleaning
    • Language dataset acquisition
    • Workshops – Train Your Team in Language AI
  • Products
    • AI Translation Platform
    • Custom Translation Portals
    • For Trados
    • For Smartling
    • For memoQ
    • Shopware Translation Plugin
    • API
    • Documentation
  • Resources
    • Blog
    • Case Studies
    • Events and Webinars
      • GenAI in Localization
    • MT Leaders
  • About Us
    • About Us
    • Terms and Conditions
    • Privacy Policy
  • Book a Call
  • Sign in
  • Home
    • For Localization Teams
    • For LSP
    • For Product Managers
    • For Translators
  • Services
    • Machine Translation Model Fine-Tuning
    • Machine Translation Evaluation
    • On-Premise Machine Translation
    • Translation Memory (TMX) Cleaning
    • Language dataset acquisition
    • Workshops – Train Your Team in Language AI
  • Products
    • AI Translation Platform
    • Custom Translation Portals
    • For Trados
    • For Smartling
    • For memoQ
    • Shopware Translation Plugin
    • API
    • Documentation
  • Resources
    • Blog
    • Case Studies
    • Events and Webinars
      • GenAI in Localization
    • MT Leaders
  • About Us
    • About Us
    • Terms and Conditions
    • Privacy Policy
  • Book a Call
  • Sign in