Identifying language origin of words

• US 20070219777A1
• Filed: 03/20/2006
• Published: 09/20/2007
• Est. Priority Date: 03/20/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chen, Yi, He, Xiaodong, Kuo, Shiun-Zu, Gong, Yifan, Chu, Min, Riley, Megan, Feige, Kevin
• Granted Patent: US 8,185,376 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/263 Language identification

Abstract

The language of origin of a word is determined by analyzing non-uniform letter sequence portions of the word.

Claims

1. A method for determining a language of origin of a word comprising analyzing non-uniform letter sequence portions of the word.

2. The method of claim 1 wherein analyzing comprising:

segmenting the word into a string of letter chunks, the letter chunks being of non-uniform length of one or more letters;

ascertaining the probability of the word belonging to a selected language by using a N-gram model based on letter chunks; and

outputting a hypothesis of the language of origin of the word based on the ascertained probability.

3. The method of claim 2 wherein the step of ascertaining includes using an N-gram model based on at least one of MI (Mutual Information) and MDL (Minimum Description Length) letter chunk.

4. The method of claim 2 wherein the step of ascertaining includes using an N-gram model based on LZ (Lempel-Ziv) letter chunk.

5. The method of claim 2 wherein the step of ascertaining includes using an N-gram model based on syllable-based letter chunk.

6. The method of claim 5 wherein the step of ascertaining includes using an N-gram model based on syllables from a closed set.

7. The method of claim 6 wherein the step of ascertaining includes using a plurality of N-gram models based on letter chunks extracted with different criteria.

8. The method of claim 7 wherein the step of ascertaining includes, for each of a plurality of selected languages, using a plurality of N-gram models based on letter chunks extracted with different criteria.

9. The method of claim 8 wherein outputting the hypothesis of the language of origin includes outputting the hypothesis based on more than one probability associated with the N-gram models based on letter chunks extracted with different criteria.

10. The method of claim 9 wherein outputting includes using a classifier.

11. A computer-readable medium having computer-executable instructions that when executed by a computer perform steps to form a hypothesis of a language of origin of a word, the instructions comprising:

segmenting the word into a string of letter chunks, the letter chunks being of non-uniform length of one or more letters;

calculating a plurality of probabilities, each probability being an indication of the word belonging to a different selected language by using a N-gram model based on letter chunks; and

outputting a hypothesis of the language of origin of the word based on the ascertained probabilities.

12. The computer-readable medium of claim 11 wherein the step of calculating includes using a plurality of N-gram models based on different criteria for letter chunk.

13. The computer-readable medium of claim 12 wherein the step of calculating includes using a plurality of N-gram models based on different criteria for letter chunks for each language.

14. The computer-readable medium of claim 11 wherein the step of calculating includes using an N-gram model based on syllable-based letter chunk.

15. The computer-readable medium of claim 13 wherein the step of calculating includes using a list of selected syllables for each language.

16. The computer-readable medium of claim 13 wherein the step of calculating includes using an N-gram model based on a language having a closed set of syllables.

17. A computer-readable medium having computer-executable instructions that when executed by a computer perform steps to process language, the instructions comprising:

determining a language of origin for a word based on analyzing non-uniform letter sequence portions of the word; and

outputting information pertaining to the determined language of origin for the word.

18. The computer-readable medium of claim 17 wherein determining comprises:

segmenting the word into a string of letter chunks, the letter chunks being of non-uniform length of one or more letters;

calculating a plurality of probabilities, each probability being an indication of the word belonging to a different selected language by using a N-gram model based on letter chunks; and

outputting a hypothesis of the language of origin of the word based on the calculated probabilities.

19. The computer-readable medium of claim 17 wherein the step of determining includes using a plurality of N-gram models based on different criteria for letter chunks.

20. The computer-readable medium of claim 19 wherein the executable instructions comprise one of a speech recognizer, a speech synthesizer, a speech application using grammars and the machine translator.