Techniques for transliterating input text from a first character set to a second character set

• US 9,613,029 B2
• Filed: 02/28/2012
• Issued: 04/04/2017
• Est. Priority Date: 02/28/2012
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Yang, Fan, Buryak, Kirill, Yuan, Feng, Liao, Baohua
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): KIM, JONATHAN C
• Application Number:
  US14/381,395
• Publication Number:
  US 20150088487A1
• Time in Patent Office: 1,862 Days
• Field of Search: None
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 40/53 Processing of non-Latin text kana-to-kanji conversion G06F40/129; vowelisation G06F40/232
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

Computer implemented techniques for performing transliteration of input text in a first character set to a second character set are disclosed. The techniques include receiving input text and determining a set of possible transliterations of the input text based on a plurality of mapping standards. Each mapping standard defines a mapping of characters in the first character set to characters in the second character set. The techniques further include determining a set of candidate words in the target language based on the possible transliterations and a text corpus. The techniques also include determining a likelihood score for each one of the candidate words based on a language model in the target language previously received words. The techniques also include providing one or more candidate words based on the likelihood scores and receiving a user selection indicating one of the candidate words.

Claims

1. A computer-implemented method comprising:

receiving, at a computing device having one or more processors and from a user via a keyboard, input text in a first character set via an input method editor;

determining, at the computing device, a set of possible transliterations of the input text based on a plurality of mapping standards, each possible transliteration of the set of possible transliterations corresponding to a transliteration of the input text into a second character set corresponding to a target language, each mapping standard of the plurality of mapping standards defining a mapping of each and every character in the second character set to one or more characters in the first character set, and each mapping standard having an associated transliteration probability stored for use with the input method editor, each transliteration probability being indicative of a likelihood that its corresponding mapping standard is appropriate for transliterating text from the user in the first character set to the second character set;

determining a transliteration score for each of the possible transliterations based on the transliteration probabilities, the transliteration score being indicative of a likelihood that its corresponding possible transliteration is an accurate transliteration of the input text;

determining, at the computing device, a set of candidate words in the target language based on the set of possible transliterations and a text corpus of the target language, wherein the set of candidate words includes words in the text corpus that match one of the set of possible transliterations, that are similar to one of the set of possible transliterations, and sound similar to one of the set of possible transliterations;

determining, at the computing device, a likelihood score for each one of the set of candidate words based on a language model in the target language and one or more previous words received, each likelihood score being indicative of a probability that a corresponding candidate word corresponds to the input text;

providing, from the computing device, one or more candidate words of the set of candidate words based on the likelihood scores and the transliteration scores;

receiving a user selection indicating one of the candidate words;

monitoring, at the computing device, tendencies of the user by determining a particular mapping standard of the plurality of mapping standards on which the selected candidate word was based; and

adjusting, at the computing device, the transliteration probabilities stored for use with the input method editor based on the tendencies of the user as determined from the particular mapping standard.

2. A computer-implemented method comprising:

receiving, at a computing device having one or more processors and from a user via a keyboard, input text in a first character set via an input method editor;

determining, at the computing device, a set of possible transliterations of the input text based on a plurality of mapping standards, each possible transliteration of the set of possible transliterations corresponding to a transliteration of the input text into a second character set corresponding to a target language, each mapping standard of the plurality of mapping standards defining a mapping of each and every character in the second character set to one or more characters in the first character set and each mapping standard having an associated transliteration probability stored for use with the input method editor, each transliteration probability being indicative of a likelihood that its corresponding mapping standard is appropriate for transliterating text from the user in the first character set to the second character set;

determining, at the computing device, a transliteration score for each of the possible transliterations based on the transliteration probabilities, the transliteration score being indicative of a likelihood that its corresponding possible transliteration is an accurate transliteration of the input text;

determining, at the computing device, a set of candidate words in the target language based on the set of possible transliterations and a text corpus of the target language, the text corpus corresponding to a set of known words in the target language;

determining, at the computing device, a likelihood score for each one of the set of candidate words based on a language model in the target language and one or more previous words received, each likelihood score being indicative of a probability that a corresponding candidate word corresponds to the input text;

providing, from the computing device, to a user device one or more candidate words of the set of candidate words based on the likelihood scores, the transliteration probabilities, and the transliteration scores;

receiving, at the computing device, a user selection indicating one of the candidate words;

monitoring, at the computing device, tendencies of the user by determining a particular mapping standard of the plurality of mapping standards on which the selected candidate word was based; and

adjusting, at the computing device, the transliteration probabilities stored for use with the input method editor based on the tendencies of the user as determined from the particular mapping standard.

3. The computer-implemented method of claim 2, wherein the text corpus includes at least one dictionary listing a plurality of words in the target language.

4. The computer-implemented method of claim 3, further comprising determining a degree of similarity between one of the possible transliterations and one of the words in the text corpus and selectively including the one word in the set of candidate words based on the degree of similarity.

5. The computer-implemented method of claim 2, wherein determining the set of candidate words further comprises including words in the text corpus that sound similar to one or more of the possible transliterations in the set of candidate words.

6. The computer-implemented method of claim 5, wherein the words that sound similar to one or more possible transliterations are determined by a Soundex algorithm.

7. The computer-implemented method of claim 2, wherein the language model is a Hidden Markov Model.

8. The computer-implemented method of claim 2, wherein the user device is the computing device.

9. The computer-implemented method of claim 2, wherein the input text is in a source language and the method further comprises:

comparing the input text with a text corpus in the source language; and

when the input text is found in the text corpus in the source language, including the input text in the set of candidates words.

10. The computer-implemented method of claim 2, wherein the transliteration probabilities are initially assigned based on a relative popularity of each mapping standard.

11. A computing device, comprising:

a keyboard that receives user input from a user indicating input text in a first character set via an input method editor;

a transliteration determination module configured to (i) receive the input text and determine a set of possible transliterations of the input text based on a plurality of mapping standards and the input text, each possible transliteration of the set of possible transliterations corresponding to a transliteration of the input text into a second character set corresponding to a target language, each mapping standard of the plurality of mapping standards defining a mapping of each and every character in the second character set to one or more characters in the first character set and each mapping standard having an associated transliteration probability stored for use with the input method editor, each transliteration probability being indicative of a likelihood that its corresponding mapping standard is appropriate for transliterating text from the user in the first character set to the second character set, and (ii) determine a transliteration score for each of the possible transliterations based on the transliteration probabilities, the transliteration score being indicative of a likelihood that its corresponding possible transliteration is an accurate transliteration of the input text;

a candidate word determination module configured to determine a set of candidate words in the target language based on the set of possible transliterations and a text corpus of the target language, the text corpus corresponding to a set of known words in the target language;

a word selection module configured to;

a) determine a likelihood score for each candidate in the set of candidate words based on a language model in the target language and one or more previous words received, each likelihood score being indicative of a probability that its corresponding candidate word corresponds to the input text,

b) provide one or more candidate words of the set of candidate words based on the likelihood scores, the transliteration probabilities of the one or more candidate words, and the transliteration scores, and

c) receive a user selection indicating one of the candidate words; and

a feedback module configured to;

monitor tendencies of the user by determining a particular mapping standard of the plurality of mapping standards on which the selected candidate word was based; and

adjust the transliteration probabilities stored for use with the input method editor based on the tendencies of the user as determined from the particular mapping standard.

12. The computing device of claim 11, wherein the text corpus includes at least one dictionary listing a plurality of words in the target language.

13. The computing device of claim 12, wherein the candidate word determination module is further configured to determine a degree of similarity between one of the possible transliterations and one of the words in the text corpus and to selectively include the one word based on the degree of similarity.

14. The computing device of claim 11, wherein the candidate word determination module is further configured to include words in the text corpus that sound similar to one or more of the possible transliterations.

15. The computing device of claim 14, wherein the candidate word determination module determines the words that sound similar to one or more possible transliterations based on a Soundex algorithm.

16. The computing device of claim 11, wherein the language model is a Hidden Markov Model.

17. The computing device of claim 11, wherein the input text is in a source language and the candidate word determination module is further configured to:

compare the input text with a text corpus in the source language; and

when the input text is found in the text corpus in the source language, include the input text in the set of candidates words.

18. The computing device of claim 11, wherein the transliteration probabilities are initially assigned based on a relative popularity of each mapping standard.