What is an example of zero-shot learning in machine translation?

Zero-shot learning in machine translation occurs when a model translates between a language pair it was never explicitly trained on. For example, imagine a multilingual model trained to translate English to French and English to German but not French to German directly. If the model can still handle French-to-German translation by leveraging what it learned from the overlapping English-centric data, that’s zero-shot learning. This approach avoids the need to train separate models for every possible language pair, which is especially useful for supporting less common languages or new combinations.

A concrete example is Google’s multilingual neural machine translation (MNMT) system. This model is trained on many language pairs (e.g., English-Spanish, English-Japanese) but can also translate between Spanish and Japanese without direct training data for that pair. The key is how the model represents languages. Instead of treating each language as isolated, it uses a shared embedding space where all languages are mapped to a common intermediate representation. During training, the encoder learns to convert any input language into this shared space, and the decoder learns to generate any target language from it. This allows the model to generalize to unseen language pairs by combining the encoder for one language with the decoder for another, even if they were never paired in training data.

However, zero-shot performance often lags behind supervised translation for seen language pairs. For instance, translating Spanish to Japanese via the shared embeddings might produce grammatical output but could include subtle errors due to missing cultural nuances or syntactic differences not captured in the shared space. To mitigate this, some systems use techniques like language tokens (e.g., adding <es> to indicate Spanish input and <ja> for Japanese output) to explicitly guide the model. While not perfect, zero-shot translation is practical for scenarios where training dedicated models is infeasible, such as supporting hundreds of low-resource languages. Developers can implement similar approaches using frameworks like Hugging Face’s Transformers, which support multilingual models like mBART or M2M-100 that enable zero-shot capabilities out of the box.