Selection and use of nonstatistical translation components in a statistical machine translation framework

• US 8,666,725 B2
• Filed: 04/15/2005
• Issued: 03/04/2014
• Est. Priority Date: 04/16/2004
• Status: Active Grant

• Current Assignee: University of Southern California
• Original Assignee: University of Southern California
• Inventors: Och, Franz Josef
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US11/107,304
• Publication Number:
  US 20060015320A1
• Time in Patent Office: 3,245 Days
• Field of Search: 704/2, 704/3, 704/4, 704/5, 704/9, 715/536
• US Class Current: 704/4
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

A system with a nonstatistical translation component integrated with a statistical translation component engine. The same corpus may be used for training the statistical engine and also for determining when to use the statistical engine and when to use the translation component. This training may use probabilistic techniques. Both the statistical engine and the translation components may be capable of translating the same information, however the system determines which component to use based on the training. Retraining can be carried out to add additional components, or when after additional translator training.

Claims

1. A computer-implemented method for integrating non-statistical translators with statistical translators, the method comprising:

training a machine translation system when to use only a statistical translator by applying a machine learning method to a bilingual text that has been annotated with the output of a non-statistical translation component along with information identifying the type of the translation component; and

training the machine translation system when to use only a non-statistical translator by applying a machine learning method to a bilingual text that has been annotated with the output of a statistical translator along with information identifying the type of the translator, wherein the statistical translator and the non-statistical translator are both based on the same training corpus.

2. The method as in claim 1, further comprising selecting whether to use the statistical translator to translate at least a portion of an unknown text, or whether to use the non-statistical translator to translate another portion of the unknown text, said selecting whether to use the statistical translator or the non-statistical translator also being based on said training.

3. The method as in claim 1, wherein said training comprises:

determining a plurality of first phrases which can be translated by the non-statistical translator;

testing a translation of said first phrases to determine if said non-statistical translator has properly translated said first phrases; and

using information from said testing to train a classifier when to use said non-statistical translator.

4. The method as in claim 3, wherein said non-statistical translator is a translation component for proper nouns.

5. The method as in claim 3, wherein said non-statistical translator is a translation component for names.

6. The method as in claim 3, wherein said non-statistical translator is a translation component for numbers.

7. The method as in claim 3, wherein said testing comprises detecting variants of a translated phrase, and accepting said phrase as being a proper translation if it is one of said variants.

8. The method as in claim 3, further comprising annotating a training corpus based on said testing to form an annotated training corpus, with annotations that represent results from said testing.

9. The method as in claim 3, wherein said classifier is a probabilistic classifier.

10. The method as in claim 1, further comprising retraining when to use the statistical translator and the non-statistical translator, responsive to an occurrence.

11. The method as in claim 10, wherein said action comprises adding an additional non-statistical translator component.

12. The method as in claim 10, wherein said action comprises improving a translator component.

13. The method as in claim 1, further comprising training a format of an output of said machine translation system, based on said training corpus, and allowing selection of one of a plurality of different formats within said training corpus.

14. The method as in claim 1, wherein said training comprises:

selecting the statistical translator to translate a first set of expressions; and

selecting the non-statistical translator to translate a second set of expressions.

15. The method as in claim 14, wherein said training comprises:

determining that the statistical translator translates the first set of expressions correctly more often than the non-statistical translator; and

determining that the non-statistical translator translates the second set of expressions correctly more often than the statistical translator.

16. The method as in claim 1, wherein said training is completed prior to applying the method to translating a target word string.

17. A system for integrating non-statistical translators with statistical translators, the system comprising:

a statistical translation system stored in memory and executed by a processor to operate based on translation data;

a translation component formed of a non-statistical translator; and

a classifier stored in memory and executed by the processor to determine automatically when to use only said statistical translator by applying a machine learning method to a bilingual text that has been annotated with the output of a non-statistical translation component along with information identifying the type of the translation component and when to use only said non-statistical translator by applying a machine learning method to a bilingual text that has been annotated with the output of a statistical translator along with information identifying the type of the translator to translate a first phrase, when both said statistical translation system and said translation component are each capable of translating said first phrase.

18. The system as in claim 17, further comprising a training corpus for said statistical translator and for said nonstatistical non-statistical translator and also for said classifier.

19. The system as in claim 17, further comprising a training element stored in memory and executed by a processor to:

determine a plurality of phrases which can be translated by the non-statistical translator;

test a translation of said phrases to determine if said non-statistical translator has properly translated each phrase; and

use information from said testing to train said classifier when to use said non-statistical translator.

20. The system as in claim 17, wherein said non-statistical translator is a translation component for proper nouns.

21. The system as in claim 17, wherein said non-statistical translator is a translation component for names.

22. The system as in claim 17, wherein said non-statistical translator is a translation component for numbers.

23. The system as in claim 19, further comprising a variant detector stored in memory and executed by the processor to detect variants of a translated phrase, and accept said phrase as being a proper translation if it is one of said variants.

24. The system as in claim 18, wherein the training corpus is annotated based on said testing with annotations that represent results from said testing.

25. The system as in claim 17, wherein said classifier is a probabilistic classifier.

26. The system as in claim 17, further comprising at least one additional non-statistical translator component.

27. The system as in claim 18, further comprising an output module stored in memory and executed by the processor to format an output of said machine translation system, based on said training corpus.

28. The system as in claim 17, wherein:

said classifier determines to use said statistical translator when said first phrase belongs to a first set of expressions; and

said classifier determines to use said non-statistical translator when said first phrase belongs to a second set of expressions.

29. The system as in claim 28, wherein:

said statistical translator translates the first set of expressions correctly more often than the non-statistical translator; and

said non-statistical translator translates the second set of expressions correctly more often than the statistical translator.

30. The system as in claim 17, wherein said determining by said classifier is completed prior to applying the method to translating a target word string.