Unbabel

Maria Pires
Olivier Cuzacq

2020
Lead Product Designer

Unbabel is a Language Operations Platform that provides human-quality translation of customer service tickets at a fraction of the cost and time. Powered by AI and enhanced by professional linguists, Unbabel’s platform leverages a global community of translators to correct machine-translated text. This ensures human-quality results, but the cost of these translators represents a significant portion of Unbabel’s operational expenses.

  • Unbabel's translation pipeline which includes humans in the loop.

  • The interface which translators use to edit machine-translated text.

Reducing operational costs at Unbabel, through Cost Per Word (CPW)

Cost Per Word (CPW) is a key business metric calculated based on a translator's hourly rate (which varies with quality and speed), the time spent on corrections, and the number of words in a translation task. Reducing CPW without sacrificing translation quality was a primary goal. Translators were often performing unnecessary corrections, leading to wasted time and higher costs. Additionally, inconsistent translations stemmed from stylistic preferences of multiple translators working on templated customer service tickets. Addressing these inefficiencies was critical.

Using translation memories to decrease Cost Per Word (CPW)

Translation memories (TMs) — segments of previously translated text stored in a database — were underutilized. Curated TMs (reviewed by professional linguists) and uncurated TMs (automatically stored after being used three times) were not clearly indicated to translators. This led to redundant corrections and inefficiencies.

Design intervention:

  1. Highlighting Translation Memories

    • Text from TMs was highlighted in grey. On hover, a padlock icon indicated if the TM was curated (locked from editing) or uncurated (editable). This visual cue helped translators focus their efforts on correcting machine-translated text, marked in black.

  2. Error Reporting Mechanism

    • Translators could report errors in curated TMs. The frequency of reports was used to assess the quality of curated TMs, enabling continuous improvement.

  3. Visual Priority

    • The interface emphasized areas requiring correction, ensuring translators’ efforts were directed efficiently without being distracted by TMs.

  • A curated TM that has previously been edited and approved by the client.

  • An uncurated TM that has previously been edited but could contain errors.

Achieving a 15% reduction in Cost Per Word

The intervention was piloted with tasks for one client and one language pair (English to Japanese) due to the abundance of curated TMs and the lack of gender agreement complexities in Japanese. The experiment resulted in a 15% reduction in CPW. The scope was later expanded to non-gendered languages such as English to Turkish, Korean, Mandarin, and Cantonese.

A translator working on a task. Notice how the translator only makes small adjustments as needed.

Expanding the experiment to non-gendered language pairs.

95% of the total volume of tasks came from 16 language pairs. There was an opportunity to have a significant impact on CPW by adapting the experiment for gendered languages.

While early results were promising, extending the experiment to gendered languages posed unique challenges. Locking TMs could lead to gender or number agreement errors, while leaving them editable risked introducing stylistic inconsistencies.

We modified our approach to the design intervention. For gendered languages, padlock icons were replaced with shields, symbolising confidence in the translation rather than restriction. Translators could edit both curated and uncurated TMs, allowing flexibility while maintaining quality.

  • A curated TM represented with high confidence.

  • An uncurated TM represented with medium confidence.

Addressing task timing and translator behaviour

Translators were sometimes “holding” tasks to maximise earnings or rushing through tasks due to inaccurate time limits. The goal of the second part of this experiment was to eliminate this kind of behaviour, and provide fairer time limits for both the translators and Unbabel.

Below are the design interventions I made to the translators interface in order to impact CPW.

  1. Dynamic Time Estimates:

    • A new formula accounted for task length, the number of TMs, and the translator’s average speed. Faster translators were allocated less time, while slower translators received more time to complete the same task.

  2. Transparent Interface:

    • Translators were informed of the estimated time before accepting a task. A clear notification indicated when the paid time had elapsed, providing fairness and clarity.

  3. Mobile Integration:

    • The solution was adapted for both Android and iOS platforms to ensure consistency across devices.

The updated time allocation formula and interface adjustments further reduced CPW while ensuring fair compensation for translators.

  • The iOS mobile app interface

  • A modal displayed to the editor before accepting the job.

  • Translation memories displayed to the editor.

  • A warning to let the editor know the allocated time has elapsed.

Reflecting on a maturing approach to experimentation

These initiatives achieved measurable success:

  • A 15% reduction in CPW in the initial experiment, with further gains as solutions were scaled.

  • Improved translator efficiency and satisfaction through clear, transparent interfaces.

  • Enhanced team morale, demonstrating a maturing approach to experimentation and iterative design.

This project underscored the importance of balancing ambitious goals with practical constraints. Early involvement of front-end developers enabled rapid prototyping, prioritizing learnings over perfection. By aligning design and development efforts with business objectives, Unbabel successfully reduced costs while maintaining the high-quality standards its clients expect.