TRANSLITERATION USING MACHINE TRANSLATION PIPELINE

• US 20180174572A1
• Filed: 12/21/2016
• Published: 06/21/2018
• Est. Priority Date: 12/21/2016
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Pino, Juan Miguel, Ramanathan, Madhu
• Granted Patent: US 10,810,380 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/129 Handling non-Latin characters, e.g. kana-to-kanji conversion
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

Embodiments are disclosed for transliteration based on a machine translation model training pipeline. A method according to some embodiments includes steps of: training a probabilistic model for transliteration from a first script system to a second script system using a machine translation model training pipeline; segmenting, using the probabilistic model, an input string in the first script system into phonemes that correspond to characters in the second script system; converting the phonemes in the first script system into the characters in the second script system, the characters forming a word or a word prefix in the second script system; and outputting the word or the word prefix in the second script system.

Claims

1. A method for transliteration comprising:

training a probabilistic model for transliteration from a first script system to a second script system using a machine translation model training pipeline;

segmenting, using the probabilistic model, an input string in the first script system into phonemes that correspond to characters in the second script system;

converting the phonemes in the first script system into the characters in the second script system, the characters forming a word or a word prefix in the second script system; and

outputting the word or the word prefix in the second script system.

2. The method of claim 1, wherein the training comprises:

training a probabilistic model for transliteration from a first script system to a second script system using a machine translation model training pipeline based on a training data set for a machine translation model.

3. The method of claim 1, wherein the training comprises:

training a probabilistic model for transliteration from a first script system to a second script system using a machine translation model training pipeline based on a training data set for a machine translation model;

wherein the training data set includes profile names provided users in the first script system and the second script system.

4. The method of claim 1, wherein the training comprises:

training a probabilistic model for transliteration from a first script system to a second script system using a machine translation model training pipeline based on a training data set for a machine translation model;

wherein the training data set includes a user-input word in the first script system and a user-selected alternate word in the second script system.

5. The method of claim 4, wherein a user inputs the user-input word in the first script system and selects the user-selected alternate word in the second script system by using a cross-script software or hardware keyboard.

6. The method of claim 1, wherein the segmenting comprises:

determining, using a machine translation model, an alignment between phonemes of the input string in the first script system and corresponding characters of the word or the word prefix in the second script system.

7. The method of claim 1, wherein the segmenting comprises:

determining, using a machine translation model, an alignment between phonemes of the input string in the first script system and corresponding characters of the word or the word prefix in the second script system;

wherein the alignment is monotonic such that relative positions of the characters within the word or the word prefix in the second script system follow relative positions of the phonemes within the input string in the first script system.

8. The method of claim 1, wherein the segmenting comprises:

determining, using a machine translation model, an alignment between phonemes of the input string in the first script system and corresponding characters of the word or the word prefix in the second script system;

wherein the alignment is complete such that there is a one-to-one mapping between positions of the phonemes within the input string in the first script system and positions of the characters within the word or the word prefix in the second script system.

9. The method of claim 6, wherein the determining an alignment comprises:

assigning indexes to characters of the word or the word prefix in the second script system, the indexes representing positions of corresponding phonemes within input string in the first script system; and

determining, by the machine translation model, values of the indexes assigned to the characters of the word or the word prefix in the second script system.

10. The method of claim 1, wherein the training comprises:

training a probabilistic model for transliteration from a first script system to a second script system using an expectation maximization algorithm.

11. The method of claim 10, wherein the training comprises:

training a probabilistic model for transliteration by alternating between an expectation step and a maximization step.

12. The method of claim 11, wherein the training comprises:

conducting the expectation step for training the probabilistic model by computing posterior probabilities of segmentation candidates of the input string.

13. The method of claim 12, wherein the training comprises:

conducting the maximization step for training the probabilistic model by computing a maximum likelihood estimation of a likelihood function using posterior probabilities as weights for the segmentation candidates of the input string.

14. The method of claim 11, wherein the expectation step comprises:

identifying segmentation candidates of the input string by exploring paths of a prefix tree structure, the prefix tree structure including nodes representing character groups of the input string.

15. The method of claim 11, wherein the expectation step comprises:

identifying segmentation candidates of the input string by exploring paths of a directed acyclic graph, the directed acyclic graph including nodes representing character groups of the input string and directional edges between nodes.

16. The method of claim 1, wherein the training comprises:

training a probabilistic model for transliteration from a first script system to a second script system using a machine translation model training pipeline based on a training data set;

wherein the training data set includes a parallel corpus, the parallel corpus including words in the first script system and corresponding words in the second script system.

17. A non-transitory machine-readable storage medium comprising a program containing a set of instructions for causing a machine to execute procedures for transliteration based on a machine translation model training pipeline, the procedures comprising:

training a probabilistic model for transliteration from a first script system to a second script system using a machine translation model training pipeline;

segmenting, using the probabilistic model, an input string in the first script system into phonemes that correspond to characters in the second script system;

converting the phonemes in the first script system into the characters in the second script system, the characters forming a word or a word prefix in the second script system; and

outputting the word or the word prefix in the second script system.

18. The method of claim 17, wherein the training comprises:

training a probabilistic model for transliteration using a training data set that includes profile names provided users in the first script system and the second script system.

19. The method of claim 17, wherein the training comprises:

training a probabilistic model for transliteration using a training data set that includes a user-input word in the first script system and a user-selected alternate word in the second script system, wherein a user inputs the user-input word in the first script system and selects the user-selected alternate word in the second script system by using a cross-script software or hardware keyboard.

20. A computing device, comprising:

a processor; and

a memory storing instructions which, when executed by the processor, cause the computing device to perform a process including;

training a probabilistic model for transliteration from a first script system to a second script system using a machine translation model training pipeline;

segmenting, using the probabilistic model, an input string in the first script system into phonemes that correspond to characters in the second script system;

converting the phonemes in the first script system into the characters in the second script system, the characters forming a word or a word prefix in the second script system; and

outputting the word or the word prefix in the second script system.