Method and apparatus for training transliteration model and parsing statistic model, method and apparatus for transliteration

• US 7,853,444 B2
• Filed: 10/06/2006
• Issued: 12/14/2010
• Est. Priority Date: 10/09/2005
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Wang, Haifeng, Guo, Yuqing
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US11/539,435
• Publication Number:
  US 20070124133A1
• Time in Patent Office: 1,530 Days
• Field of Search: None
• US Class Current: 704/2
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

The present invention provides a method and apparatus for training a parsing statistic model, a method and apparatus for transliteration. Said parsing statistic model is to be used in transliteration between a single-syllable language and a multi-syllable language and includes sub-syllable parsing probabilities of said multi-syllable language. Said method for training the parsing statistic model comprising: inputting a bilingual proper name list as corpus, said bilingual proper name list includes a plurality of proper names of said multi-syllable language and corresponding proper names of said single-syllable language respectively; parsing each of said plurality of proper names of multi-syllable language in said bilingual proper name list into a sub-syllable sequence using parsing rules; determining whether said parsing is correct according to the corresponding proper name of said single-syllable language in said bilingual proper name list; and training said parsing statistic model base on the result of parsing that is determined as correct.

Claims

1. An apparatus for training a parsing statistic model, which is to be used in transliteration between a single-syllable language and a multi-syllable language and includes sub-syllable parsing probabilities of said multi-syllable language, comprising:

a corpus inputting unit configured to input a bilingual proper name list as corpus, said bilingual proper name list includes a plurality of proper names of said multi-syllable language and corresponding proper names of said single-syllable language respectively;

a rule parsing unit configured to parse said plurality of proper names of multi-syllable language in said bilingual proper name list into sub-syllable sequences using parsing rules;

a parsing determining unit configured to determine whether a parsing of said proper name of multi-syllable language is correct according to the corresponding proper name of said single-syllable language in said bilingual proper name list; and

a parsing statistic model training unit configured to train said parsing statistic model base on the result of parsing that is determined as correct.

2. The apparatus for training a parsing statistic model according to claim 1, wherein said parsing determining unit is configured to determine whether the number of sub-syllables in said sub-syllable sequence parsed from said proper name of said multi-syllable language is equal to the number of syllables in said corresponding proper name of single-syllable language in said bilingual proper name list, and if it is equal, the parsing is determined as correct, otherwise the parsing is determined as incorrect.

3. The apparatus for training a parsing statistic model according to claim 1 or 2, further comprising:

a model parsing unit configured to parse the proper name of multi-syllable language that is determined as incorrectly parsed again using said parsing statistic model.

4. The apparatus for training a parsing statistic model according to claim 1, wherein said parsing statistic model training unit further comprises:

a parsing probability calculator configured to calculate an occurrence probability of each pair of adjacent sub-syllables based on the adjacent relationships among sub-syllables in the sub-syllable sequences parsed from said proper names of multi-syllable language that are determined as correct.

5. An apparatus for transliteration from a single-syllable language to a multi-syllable language, comprising:

a syllable sequence obtaining unit configured to obtain a syllable sequence corresponding to a word of said single-syllable language to be transliterated;

a transliteration model including translation relationships between syllables of said single-syllable language and sub-syllables of said multi-syllable language and their translation probabilities respectively;

a sub-syllable translating unit configured to obtain at least one sub-syllable of said multi-syllable language corresponding to each syllable in said syllable sequence obtained by said syllable sequence obtaining unit and its translation probability by using said transliteration model;

a parsing statistic model including sub-syllable parsing probabilities of said multi-syllable language;

a searching unit configured to search for a sub-syllable sequence having the highest probability corresponding to said syllable sequence as a transliteration result based on said parsing statistic model and said at least one sub-syllable of said multi-syllable language corresponding to each syllable in said syllable sequence and its translation probability.

6. The apparatus for transliteration from a single-syllable language to a multi-syllable language according to claim 5 further comprising:

a pronunciation lexicon including character pronunciations of said single-syllable language;

wherein said syllable sequence obtaining unit is configured to obtain a syllable sequence corresponding to a word of said single-syllable language based on said pronunciation lexicon.

7. The apparatus for transliteration from a single-syllable language to a multi-syllable language according to claim 5, wherein said searching unit is configured to calculating with a search algorithm to find the sub-syllable sequence having the highest probability based on said at least one sub-syllable of said multi-syllable language corresponding to each syllable in said syllable sequence and its translation probability, and the sub-syllable parsing probabilities in said parsing statistic model.

8. The apparatus for transliteration from a single-syllable language to a multi-syllable language according to claim 7, wherein said search algorithm is any one or a combination of depth first search, breadth first search, A* search and Viterbi algorithm.

9. An apparatus for transliteration from a multi-syllable language to a single-syllable language, comprising:

a sub-syllable parsing unit configured to parse a word of said multi-syllable language that needs to be transliterated into a sub-syllable sequence;

a transliteration model including translation relationships between syllables of said single-syllable language and sub-syllables of said multi-syllable language and their translation probabilities respectively;

a syllable translating unit configured to obtain at least one syllable of said single-syllable language corresponding to each sub-syllable in said sub-syllable sequence and its translation probability according to said transliteration model;

a character translating unit configured to obtain a character corresponding to each said syllable of single-syllable language;

a language model including character adjacent probabilities of said single-syllable language;

a searching unit configured to search for a character sequence having the highest probability corresponding to said sub-syllable sequence as a transliteration result based on said language model and said at least one syllable of said single-syllable language corresponding to each sub-syllable in said sub-syllable sequence and its translation probability obtained by said syllable translating unit.

10. The apparatus for transliteration from a multi-syllable language to a single-syllable language according to claim 9, wherein said sub-syllable parsing unit further comprises:

a parsing statistic model including sub-syllable parsing probabilities of said multi-syllable language;

a model parsing unit configured to calculate to find a sub-syllable sequence having the highest probability corresponding to said word of multi-syllable language using a search algorithm based on said parsing statistic model.

11. The apparatus for transliteration from a multi-syllable language to a single-syllable language according to claim 10, wherein said sub-syllable parsing unit further comprises:

a rule parsing unit configured to parse said word of multi-syllable language using parsing rules.

12. The apparatus for transliteration from a multi-syllable language to a single-syllable language according to claim 9, further comprising:

a pronunciation lexicon including character pronunciations of said single-syllable language.

13. The apparatus for transliteration from a multi-syllable language to a single-syllable language according to claim 9, wherein said searching unit is configured to calculate with a search algorithm to find the character sequence having the highest probability based on at least one character of said single-syllable language corresponding to each sub-syllable in said sub-syllable sequence and its translation probability, and the character adjacent probabilities in said language model.

14. The apparatus for transliteration from a multi-syllable language to a single-syllable language according to claim 10 or 13, wherein said search algorithm is any one or a combination of depth first search, breadth first search, A* search and Viterbi algorithm.