Statistical translation system and method for fast sense disambiguation and translation of large corpora using fertility models and sense models

• US 6,092,034 A
• Filed: 07/27/1998
• Issued: 07/18/2000
• Est. Priority Date: 07/27/1998
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: McCarley, Jeffrey Scott, Roukos, Salim
• Primary Examiner(s): Thomas, Joseph
• Application Number:
  US09/123,166
• Time in Patent Office: 722 Days
• Field of Search: 704/9, 704/2, 704/3, 704/4, 704/7, 704/1, 704/8, 704/10, 704/277, 704/251, 704/255, 704/257, 711/1, 707/530, 707/531, 707/532
• US Class Current: 704/2
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

A system and method for translating a series of source words in a first language to a series of target words in a second language is provided. The system includes an input device for inputting the series of source words. A fertility hypothesis generator operatively coupled to the input device generates at least one fertility hypotheses for a fertility of a source word, based on the source word and a context of the source word. A sense hypothesis generator operatively coupled to the input device generates sense hypotheses for a translation of the source word, based on the source word and the context of the source word. A fertility model operatively coupled to the fertility hypothesis generator determines a probability of the fertility of the source word, based on the source word and the context of the source word. A sense model operatively coupled to the sense hypothesis generator determines a probability of a target word being a correct translation of the source word, based on the source word and the context of the source word. A decoder operatively coupled to the fertility and sense models for generating a list of target words for the translation of the source word, based on the probability calculated by the fertility model and the probability calculated by the sense model.

Claims

1. A system for translating a series of source words in a first language to a series of target words in a second language, comprising:

input means for inputting the series of source words;

a fertility hypothesis generator operatively coupled to said input means for generating at least one fertility hypotheses for a fertility of a source word, based on the source word and a context of the source word;

a sense hypothesis generator operatively coupled to said input means for generating sense hypotheses for a translation of the source word, based on the source word and the context of the source word;

a fertility model operatively coupled to said fertility hypothesis generator for determining a probability of the fertility of the source word, based on the source word and the context of the source word;

a sense model operatively coupled to said sense hypothesis generator for determining a probability of a target word being a correct translation of the source word, based on the source word and the context of the source word; and

a decoder operatively coupled to said fertility and sense models for generating a list of target words for the translation of the source word, based on the probability calculated by said fertility model and the probability calculated by said sense model.

2. The system of claim 1, wherein the list of target words contains a null word, when the probability of the fertility of the source word being equal to zero is greater than the probability of the fertility of the source word being equal to a value other than zero.

3. The system of claim 1, wherein the context of the source word associated with said fertility model and said sense model includes at least one other word in the first language.

4. The system of claim 1, wherein the context of the source word associated with said fertility model and said sense model is a preceding and a following word of the source word.

5. The system of claim 1, wherein said fertility model is a deleted interpolation 4-gram model.

6. The system of claim 5, wherein the deleted interpolation 4-gram model has been symmetrized with respect to left and right contexts.

7. The system of claim 1, wherein said sense model utilizes an average from a maximum-entropy trigram model for a left context and another maximum-entropy trigram model for a right context.

8. The system of claim 1, further comprising a source language pre-processor operatively coupled between said input means and said fertility and sense hypothesis generators, for pre-processing the series of source words.

9. The system of claim 8, wherein the pre-processing of the series of source words comprises at least one of tokenizing, applying part of speech tags, determining morphological root words, and correcting errors in diacritical marks.

10. The system of claim 1, further comprising a target language post-processor operatively coupled to said decoder for at least one of conjugation of root words, declension of root words, and formatting of the target words output from the system.

11. The system of claim 1, wherein the at least one fertility hypotheses generated by said fertility hypothesis generator is an integer.

12. The system of claim 1, wherein the sense hypotheses generated by said sense hypothesis generator correspond to a pre-selected list of common translations of each source language word.

13. The system of claim 1, wherein the system is trained on a sentence-aligned training text including a plurality of sentences in the source language and a plurality of sentences in the target language, wherein each sentence in the plurality of source language sentences is a translation of a corresponding sentence in the plurality of target language sentences, and a training portion of the system comprises:

a word-alignment model for determining an alignment between the words of each of the plurality of source language sentences and the corresponding target language sentence;

a fertility-context event accumulator for accumulating fertility-context events from the alignments determined by said word-alignment model for at least some occurrences of the source word in a source language sentence, each fertility-context event including a number of target language words that the source language word is aligned to, the source language word, and the context of the source-language word;

a sense-context event accumulator for accumulating sense-context events from the alignments determined by said word-alignment model for at least some occurrences of the source word in a source language sentence, each sense-context event including the target word, the source-language word to which the target language word is aligned, and the context of the source-language word; and

wherein said fertility model and said sense model are constructed using the fertility-context events and sense-context events accumulated by said fertility-context event accumulator and said sense-context event accumulator, respectively.

14. The system of claim 13, wherein a confidence measure is used to exclude unreliable fertility-context and sense-context events from being accumulated by said fertility-context and sense-context event accumulators, respectively.

15. The system of claim 13, wherein the training portion of the system further comprises:

a source language pre-processor operatively coupled to said word-alignment model for pre-processing the plurality of sentences in the source language.

16. The system of claim 15, wherein the pre-processing of the plurality of sentences in the source language comprises at least one of tokenizing, applying part of speech tags, determining morphological root words, and correcting errors in diacritical marks.

17. The system of claim 13, wherein the training portion of the system further comprises:

a target language pre-processor operatively coupled to said word-alignment model for pre-processing the plurality of sentences in the target language.

18. The system of claim 17, wherein the pre-processing of the plurality of sentences in the target language comprises at least one of tokenizing, applying part of speech tags, determining morphological root words, and correcting errors in diacritical marks.

19. A method for translating a series of source words in a first language to a series of target words in a second language, comprising the steps of:

inputting the series of source words;

generating at least one fertility hypotheses for a fertility of a source word, based on a source word and a context of the source word;

generating sense hypotheses for a translation of the source word, based on the source word and the context of the source word;

determining a probability of a fertility of a source word, based on the source word and the context of the source word;

determining a probability of a target word being a correct translation of the source word, based on the source word and the context of the source word; and

generating a list of target words for the translation of the source word, based on the probability of the fertility and the probability of the target word.

20. The method of claim 19, wherein the list of target words contains a null word, when the probability of the fertility of the source word being equal to zero is greater than the probability of the fertility of the source word being equal to a value other than zero.

21. The method of claim 19, wherein the context of the source word in said determining steps includes at least one other word in the first language.

22. The method of claim 19, wherein the context of the source word in said determining steps is a preceding and a following word of the source word.

23. The method of claim 19, wherein the probability of the fertility of the source word is determined using a deleted interpolation 4-gram model.

24. The method of claim 23, wherein the deleted interpolation 4-gram model has been symmetrized with respect to left and right contexts.

25. The method of claim 19, wherein the probability of the target word is determined using an average from a maximum-entropy trigram model for a left context and another maximum-entropy trigram model for a right context.

26. The method of claim 19, further comprising the step of pre-processing the series of source words before generating the at least one fertility hypotheses and the sense hypothesis.

27. The method of claim 26, wherein said pre-processing step further comprises at least one of tokenizing, applying part of speech tags, determining morphological root words, and correcting errors in diacritical marks.

28. The method of claim 19, further comprising the step of post-processing the list of target words.

29. The method of claim 28, wherein said post-processing step comprises one of conjugation of root words, declension of root words, and formatting the target words.

30. The method of claim 19, wherein the at least one generated fertility hypotheses is an integer.

31. The method of claim 19, wherein the generated sense hypotheses correspond to a pre-selected list of common translations of each source language word.

32. The method of claim 19, wherein the method is trained on a sentence-aligned training text including a plurality of sentences in the source language and a plurality of sentences in the target language, wherein each sentence in the plurality of source language sentences is a translation of a corresponding sentence in the plurality of target language sentences, and a training portion of the method comprises:

determining an alignment between the words of each of the plurality of source language sentences and the corresponding target language sentence;

accumulating fertility-context events from the alignments for at least some occurrence of the source word in a source language sentence, each fertility-context event including a number of target language words that the source language word is aligned to, the source language word, and the context of the source-language word;

accumulating sense-context events from the alignments for at least some occurrence of the source word in a source language sentence, each sense-context event including the target word, the source-language word to which the target language word is aligned, and the context of the source-language word; and

wherein the probability of the fertility of the source word and the probability of the target are determined using the accumulated fertility-context and sense-context events, respectively.

33. The method of claim 32, further comprising the step of implementing a confidence measure to exclude unreliable fertility-context and sense-context events from being accumulated.

34. The method of claim 32, wherein the training portion further comprises the step of:

pre-processing the plurality of sentences in the source language.

35. The method of claim 34, wherein the pre-processing of the plurality of sentences in the source language comprises at least one of tokenizing, applying part of speech tags, determining morphological root words, and correcting errors in diacritical marks.

36. The method of claim 32, wherein the training portion further comprises the step of:

pre-processing the plurality of sentences in the target language.

37. The method of claim 36, wherein the pre-processing of the plurality of sentences in the target language comprises at least one of tokenizing, applying part of speech tags, determining morphological root words, and correcting errors in diacritical marks.