Google Neural Machine Translation

Part of a series on
Translation
Types
Theory
Technologies
Localization
Institutional
Related topics

Google Neural Machine Translation (GNMT) was a neural machine translation (NMT) system developed by Google and introduced in November 2016 that used an artificial neural network to increase fluency and accuracy in Google Translate.[1][2][3][4] The neural network consisted of two main blocks, an encoder and a decoder, both of LSTM architecture with 8 1024-wide layers each and a simple 1-layer 1024-wide feedforward attention mechanism connecting them.[4][5] The total number of parameters has been variously described as over 160 million,[6] approximately 210 million,[7] 278 million[8] or 380 million.[9] It used WordPiece tokenizer, and beam search decoding strategy. It ran on Tensor Processing Units.

By 2020, the system had been replaced by another deep learning system based on a Transformer encoder and an RNN decoder.[10]

GNMT improved on the quality of translation by applying an example-based (EBMT) machine translation method in which the system learns from millions of examples of language translation.[2] GNMT's proposed architecture of system learning was first tested on over a hundred languages supported by Google Translate.[2] With the large end-to-end framework, the system learns over time to create better, more natural translations.[1] GNMT attempts to translate whole sentences at a time, rather than just piece by piece.[1] The GNMT network can undertake interlingual machine translation by encoding the semantics of the sentence, rather than by memorizing phrase-to-phrase translations.[2][11]

  1. ^ a b c Barak Turovsky (November 15, 2016), "Found in translation: More accurate, fluent sentences in Google Translate", Google Blog, retrieved January 11, 2017
  2. ^ a b c d Mike Schuster; Melvin Johnson; Nikhil Thorat (November 22, 2016), "Zero-Shot Translation with Google's Multilingual Neural Machine Translation System", Google Research Blog, retrieved January 11, 2017
  3. ^ Gil Fewster (January 5, 2017), "The mind-blowing AI announcement from Google that you probably missed", freeCodeCamp, retrieved January 11, 2017
  4. ^ a b Wu, Yonghui; Schuster, Mike; Chen, Zhifeng; Le, Quoc V.; Norouzi, Mohammad (2016). "Google's neural machine translation system: Bridging the gap between human and machine translation". arXiv:1609.08144. Bibcode:2016arXiv160908144W. {{cite journal}}: Cite journal requires |journal= (help)
  5. ^ "Peeking into the neural network architecture used for Google's Neural Machine Translation".
  6. ^ Qin, Minghai; Zhang, Tianyun; Sun, Fei; Chen, Yen-Kuang; Fardad, Makan; Wang, Yanzhi; Xie, Yuan (2021). "Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting". arXiv:2112.10930 [cs.NE].
  7. ^ "Compression of Google Neural Machine Translation Model – NLP Architect by Intel® AI Lab 0.5.5 documentation".
  8. ^ Langroudi, Hamed F.; Karia, Vedant; Pandit, Tej; Kudithipudi, Dhireesha (2021). "TENT: Efficient Quantization of Neural Networks on the tiny Edge with Tapered FixEd PoiNT". arXiv:2104.02233 [cs.LG].
  9. ^ "Data Augmentation | How to use Deep Learning when you have Limited Data". May 19, 2021.
  10. ^ "Recent Advances in Google Translate". research.google. Retrieved May 8, 2024.
  11. ^ Boitet, Christian; Blanchon, Hervé; Seligman, Mark; Bellynck, Valérie (2010). "MT on and for the Web" (PDF). Archived from the original (PDF) on March 29, 2017. Retrieved December 1, 2016.