Statistical method and apparatus for learning translation relationships among phrases

• US 20040098247A1
• Filed: 11/20/2002
• Published: 05/20/2004
• Est. Priority Date: 11/20/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Moore, Robert C.
• Granted Patent: US 7,249,012 B2
• Time in Patent Office: Days
• US Class Current: 704/4
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/45 Example-based machine translation; Alignment

Abstract

The present invention learns phrase translation relationships by receiving a parallel aligned corpus with phrases to be learned identified in a source language. Candidate phrases in a target language are generated and an inside score is calculated based on word association scores for words inside the source language phrase and candidate phrase. An outside score is calculated based on word association scores for words outside the source language phrase and candidate phrase. The inside and outside scores are combined to obtain a joint score.

Claims

1. A method of identifying a translation relationship between a phrase in a source language and a phrase in a target language, comprising:

receiving access to an aligned pair of multiple word units, one being a source unit in the source language and another being a target unit in the target language, the source language phrase being identified in the source unit;

generating at least one candidate phrase in the target unit, the candidate phrase being a hypothesized translation of the source language phrase;

calculating a score for each candidate phrase including an inside component based on associations between words inside the source language phrase and candidate phrase and an outside component based on associations between words outside the source language phrase and candidate phrase; and

identifying the translation relationship between the source language phrase and candidate phrase based on the score.

2. The method of claim 1 wherein pairs comprise a word in the source unit and a word in the target unit have individual word association scores indicative of a degree of association between pairs of words in the source and target units.

3. The method of claim 2 wherein calculating a score comprises:

calculating an inside score indicative of the degree of association between words in the source language phrase and words in the candidate phrase; and

calculating an outside score indicative of the degree of association between words in the source unit outside the source language phrase and words in the target unit outside the candidate phrase.

4. The method of claim 3 wherein calculating a score comprises:

combining the inside and outside scores to obtain a joint score.

5. The method of claim 4 wherein identifying the translation relationship comprises:

identifying the translation relationship between the source language phrase and the candidate phrase based on the joint score.

6. The method of claim 1 wherein generating at least one candidate phrase comprises:

generating the candidate phrase based on the individual word association scores.

7. The method of claim 5 wherein each word in the source unit has a word association indicative of a word in the target unit, including a null word, that is most strongly associated with it, and each word in the target unit has a word association indicative of a word in the source unit, including a null word, that is most strongly associated with it, and wherein receiving access comprises:

generating counts of occurrences of most strongly associated word pairs.

8. The method of claim 5 and further comprising:

detecting a capitalization pattern in the candidate phrase; and

adjusting the joint score based on the capitalization pattern.

9. The method of claim 8 wherein detecting a capitalization pattern comprises detecting at least one of:

a first pattern in which a first word in the candidate phrase begins with a capital letter;

a second pattern in which the first word in the candidate phrase does not begin with a capital letter, but one or more subsequent words in the candidate phrase begin with a capital letter; and

a third pattern in which no words in the candidate phrase begin with a capital letter.

10. The method of claim 9 wherein adjusting the joint score based on the capitalization pattern comprises:

applying a first capitalization score when the first pattern is detected;

applying a second capitalization score, when the second pattern is detected; and

applying a third capitalization score, when the third pattern is detected.

11. The method of claim 3 wherein calculating an inside score comprises:

identifying inside word pairs by;

for each word in the source language phrase, identifying a word in the candidate phrase, including a null word, that is most closely associated with it based on the word association scores; and

for each word in the candidate phrase, identifying a word in the source language phrase, including a null word, that is most closely associated with it based on the word association scores.

12. The method of claim 11 wherein calculating the inside score further comprises:

for each inside word pair, generating an inside word pair probability indicative of a probability that an occurrence of one word in the inside word pair will have another word in the inside word pair as a most strongly associated word; and

combining the inside word pair probabilities.

13. The method of claim 12 wherein calculating an outside score comprises:

identifying outside word pairs by;

for each word in the source unit, outside of the source language phrase, identifying a word in the target unit outside of the candidate phrase, including a null word, that is most closely associated with it based on the word association scores; and

for each word in the target unit outside of the candidate phrase, identifying a word in the source unit outside of the source language phrase, including a null word, that is most closely associated with it based on the word association scores.

14. The method of claim 13 wherein calculating the outside score further comprises:

for each outside word pair, generating an outside word pair probability indicative of a probability that an occurrence of one word in the outside word pair will have another word in the outside word pair as a most strongly associated word; and

combining the outside word pair probabilities.

15. The method of claim 14 wherein calculating the joint score comprises:

combining the inside and outside word pair probabilities.

16. The method of claim 1 wherein generating candidate phrases comprises:

identifying target language words in the target unit that are most strongly associated with a word in the source language phrase; and

identifying target language words in the target unit with which a word in the source language phrase is most strongly associated.

17. The method of claim 16 wherein generating candidate phrases further comprises:

generating candidate phrases as sequences of words in the target unit, constrained to those beginning and ending with an identified target language word.

18. The method of claim 17 wherein generating candidate phrases further comprises:

generating further candidate phrases that begin with a target language word beginning with a capital letter and end with an identified target language word.

19. The method of claim 5 wherein identifying the phrase translation relationship comprises:

modifying the joint scores associated with each candidate phrase based on how often throughout the corpus the candidate phrase is generated as a translation of the source language phrase based on the joint score for the candidate phrase to obtain a new joint score.

20. The method of claim 19 wherein modifying the joint score comprises:

generating counts of how often each source language phrase in the corpus is translated as a given candidate phrase based on the joint scores.

21. The method of claim 20 wherein modifying the joint score further comprises:

estimating a new inside score for each candidate phrase from the counts generated.

22. The method of claim 21 wherein modifying the joint score further comprises:

applying a scaling factor to the new inside score to obtain a scaled inside score.

23. The method of claim 22 wherein modifying the joint score further comprises:

combining the scaled inside score with the outside score for each candidate phrase to obtain the new joint score for each candidate phrase.

24. The method of claim 19 wherein identifying the phrase translation relationship comprises:

converting the new joint scores to a desired form by;

for each pair of units and each source language phrase, identifying a candidate phrase as a most likely translation of the source language phrase based on the new joint scores and assigning counts to the identified candidate phrases.

25. The method of claim 24 wherein converting comprises:

computing log-likelihood-ratio scores based on the new joint scores.

26. The method of claim 25 wherein converting further comprises:

for each unit pair, for each source language phrase, assigning counts to a best candidate phrase based on the log-likelihood-ratio scores.

27. The method of claim 26 wherein converting further comprises:

computing new log-likelihood-ratio scores based on the counts assigned.

28. The method of claim 27 wherein converting further comprises:

iterating on the steps of assigning counts and computing new log-likelihood ratio scores until the best candidate phrase stabilizes.

29. A system for identifying phrase translations in multi-word target units for identified source language phrases in multi-word source units, comprising:

an individual word association model configured to generate one or more candidate phrases and a score for each candidate phrase based on word associations between words inside the source and target language phrases and word associations between words outside the source and target language phrases.

30. The system of claim 29 wherein the source and target units are part of an aligned corpus, and further comprising:

a cross-sentence model configured to modify the score based on other candidate phrases generated for the source language phrase across the corpus, to obtain a modified score.

31. The system of claim 30 and further comprising:

a conversion model configured to convert the modified score into a desired confidence metric indicative of a confidence level associated with the candidate phrase as a translation of the source language phrase.

32. A method of generating candidate phrases in multi-word target units in a target language as hypothesized translations of an identified phrase in a multi-word source unit in a source language, comprising:

identifying first target language words in the target unit that are most strongly associated with a word in the source language phrase;

identifying second target language words in the target unit that have a word in the source language phrase that is most strongly associated it; and

generating the candidate phrases as phrases that begin and end with first or second target language words.

33. The method of claim 32 wherein generating candidate phrases further comprises:

generating additional candidate phrases as phrases that begin with words starting with a capital letter and that end with first or second target language words.