Method and system for translating information with a higher probability of a correct translation

• US 8,977,536 B2
• Filed: 06/03/2008
• Issued: 03/10/2015
• Est. Priority Date: 04/16/2004
• Status: Active Grant

• Current Assignee: University of Southern California
• Original Assignee: University of Southern California
• Inventors: Och, Franz Josef
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US12/132,401
• Publication Number:
  US 20080270109A1
• Time in Patent Office: 2,471 Days
• Field of Search: 704/2, 704/3, 704/4, 704/5, 704/9, 715/536
• US Class Current: 704/4
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

A system with a nonstatistical translation component integrated with a statistical translation component engine. The same corpus may be used for training the statistical engine and also for determining when to use the statistical engine and when to use the translation component. This training may use probabilistic techniques. Both the statistical engine and the translation components may be capable of translating the same information, however the system determines which component to use based on the training. Retraining can be carried out to add additional components, or when after additional translator training.

Claims

1. A method comprising:

executing instructions stored in memory via a processor of a language engine for;

training the language engine to train for statistical machine translation;

training the language engine when to use statistical machine translation by applying a machine learning method to a bilingual text that has been annotated with the output of a non-statistical translation component along with information identifying the type of the translation component;

translating information from a first language to a second language using at least two translation components, wherein at least one translation component is a non-statistical translation component, each of the at least two translation components capable of translating equivalent phrases, each of the at least two translation components being selected based upon evaluation of an annotated training corpus, the annotated training corpus comprising substrings in the first language that have been annotated to associate the substrings with one or more translation components that are to be utilized to translate the substrings; and

automatically selecting a preferred component from the at least two translation components, the preferred component providing a translation having a highest probability of being correct.

2. A method as in claim 1, further comprising defining a feature function that indicates when to use the at least two translation components.

3. A method as in claim 1, wherein automatically selecting the preferred component comprises:

obtaining a phrase to be translated; and

computing a probability of being correct in translating the phrase for each of the at least two translation components.

4. A method as in claim 3, further comprising:

detecting variants of a translation of the phrase; and

accepting the translation of the phrase as being proper if the translation of the phrase is equivalent to one of the variants.

5. A method as in claim 4, further comprising annotating a training corpus based on the variants to form an annotated training corpus.

6. A method as in claim 1, wherein a first component of the at least two translation components comprises a statistical translator and a second component of the at least two translation components comprises a nonstatistical translator for proper nouns.

7. A method as in claim 1, wherein a first component of the at least two translation components comprises a statistical translator and a second component of the at least two translation components comprises a nonstatistical translator for names.

8. A method as in claim 1, wherein a first component of the at least two translation components comprises a statistical translator and a second component of the at least two translation components comprises a nonstatistical translator for numbers.

9. A method as in claim 1, further comprising automatically re-selecting a second preferred component from the at least two translation components in response to an occurrence.

10. A method as in claim 9, wherein the occurrence comprises adding an additional nonstatistical translation component to the at least two translation components.

11. A method as in claim 9, wherein the occurrence comprises improving at least one of the at least two translation components.

12. A method as in claim 1, further comprising:

training a format of an output of a machine translation system based on a training corpus; and

allowing at least one of a plurality of different formats to be selected from within the training corpus.

25. The method as in claim 1, further comprising utilizing a statistical machine translator if the at least two translation components are using a rule set that is inaccurate.

26. The method as in claim 1, further comprising generating an annotated training corpus by:

applying a plurality of translation components to substrings in a first language to translate the substrings into a second language; and

annotating strings in a first language for each translation component that accurately translates the substring into a second language.

27. The method as in claim 26, further comprising evaluating context of a translation of a substring by evaluating the translated substring relative to a string from which the substring was obtained.

28. The method as in claim 27, further comprising determining an accurate translation by ensuring that the translation is both correctly translated and contextually correct.

29. The method as in claim 26, further comprising automatically regenerating the annotated training corpus when a quality of translations generated by at least one of the two translation components has increased.

30. The method as in claim 1, selecting the at least two translation components by applying a maximum entropy model to each substring that is to be translated by the at least two translation components.

13. A system comprising:

a memory for storing executable instructions;

a processor for executing the instructions for training a language engine to train for statistical machine translation and training the language engine when to use statistical machine translation by applying a machine learning method to a bilingual text that has been annotated with the output of a non-statistical translation component along with information identifying the type of the translation component;

at least two translating parts stored in memory and executable by the processor, wherein at least one translating part is a non-statistical translation part, each of the at least two translating parts operational to translate information from a first language to a second language and each of the at least two translating parts capable of translating equivalent phrases, each of the at least two translating parts being selected based upon evaluation of an annotated training corpus, the annotated training corpus comprising substrings in the first language that have been annotated to associate the substrings with one or more translating parts that are to be utilized to translate the substrings; and

a classifier part stored in memory and executable by the processor to automatically select a preferred component from the at least two translating parts, the preferred component providing a translation having a highest probability of being correct.

14. A system as in claim 13, further comprising a training corpus.

15. A system as in claim 14, further comprising an output module that formats an output based on the training corpus.

16. A system as in claim 13, further comprising a variant detector that detects variants of a translated phrase and accepts the translated phrase as being proper if the translation phrase is one of the variants.

17. A system as in claim 16, further comprising an annotated training corpus based, at least in part, on the variants.

18. A system as in claim 13, wherein a first part of the at least two translating parts comprises a statistical translator component and a second part of the at least two translating parts comprises a nonstatistical translator component for proper nouns.

19. A system as in claim 13, wherein a first part of the at least two translating parts comprises a statistical translator component and a second part of the at least two translating parts comprises a nonstatistical translator component for names.

20. A system as in claim 13, wherein a first part of the at least two translating parts comprises a statistical translator component and a second part of the at least two translating parts comprises a nonstatistical translator component for numbers.

21. A system as in claim 13, further comprising a feature part that indicates when to use the at least two translation components.

22. A system as in claim 13, wherein the classifier part comprises a probabilistic classifier.

23. A system as in claim 13, further comprising at least one additional translating part.

24. A non-transitory computer readable storage medium having embodied thereon a program, the program executable by a processor to perform a method, the method comprising:

training a language engine to train for statistical machine translation;

training the language engine when to use statistical machine translation by applying a machine learning method to a bilingual text that has been annotated with the output of a non-statistical translation component along with information identifying the type of the translation component;

translating information from a first language to a second language using at least two translation components, wherein at least one translation component is a non-statistical translation component, each of the at least two translation components capable of translating equivalent phrases, each of the at least two translation components being selected based upon evaluation of an annotated training corpus, the annotated training corpus comprising substrings in the first language that have been annotated to associate the substrings with one or more translation components that are to be utilized to translate the substrings; and

automatically selecting a preferred component from the at least two translation components, the preferred component providing a translation having a highest probability of being correct.