Use of annotations in statistical machine translation

• US 9,122,674 B1
• Filed: 12/15/2006
• Issued: 09/01/2015
• Est. Priority Date: 12/15/2006
• Status: Active Grant

• Current Assignee: SDL Inc (RWS Holdings Plc)
• Original Assignee: Language Weaver, Inc. (RWS Holdings Plc)
• Inventors: Wong, William, Bardea, Manuel
• Primary Examiner(s): ADESANYA, OLUJIMI A
• Application Number:
  US11/640,157
• Time in Patent Office: 3,182 Days
• Field of Search: 704/2
• US Class Current: 1/1
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/47 Machine-assisted translation, e.g. using translation memory

Abstract

A method, system, and computer readable medium for translating a document is provided. A statistical machine translation engine is trained using a translation memory comprising an annotation. A translation rule associated with the annotation is defined. A source document in a source language is received. The source document comprises an instance of the annotation and a string. The string is translated using the statistical machine translation engine. The instance of the annotation is processed according to the translation rule. A target document in a target language is generated based on the translated string and the processed annotation.

Claims

1. A method for translating a document, comprising:

defining translation rules associated with annotations occurring in a translation memory database, wherein the translation rules comprise an output context;

integrating the translation rules in the translation memory database into statistical machine translation used by a translation engine, the integration including training the translation engine to recognize annotations in the translation memory database while performing statistical machine translation;

training the statistical machine translation of the translation engine using both the translation memory database and parallel segments to modify statistical probability values;

storing the statistical probability values modified during the training, the stored values for use in the statistical machine translation by the translation engine;

receiving a source document in a source language, the source document comprising a string and an annotated segment of text, the annotated segment of text comprising text and an instance of an annotation;

translating the source document as a whole using the translation engine without breaking the source document into segments that are translated according to whether a match appears in the translation memory database, the translation engine configured for;

identifying annotations and associated annotated segments of the text in the source document to be processed using the translation rules, and strings in the source document to be translated using statistical machine translation,

translating the identified strings in the source document using the statistical machine translation, and

processing the identified instances of the annotations and the associated annotated segments of text according to the translation rules using the translation memory database, at least one of the translation rules being associated with the annotation; and

generating a target document in a target language based on the translated source document and the translated text associated with the annotation.

2. The method recited in claim 1, wherein the annotation indicates a word class.

3. The method recited in claim 2, wherein the word class is associated with a value.

4. The method recited in claim 2, wherein the output context is associated with the word class.

5. The method recited in claim 1, wherein the annotation indicates a boundary.

6. The method recited in claim 5, wherein translating the text associated with the instance of the annotation comprises translating text occurring after the annotation separately from text occurring before the annotation.

7. The method recited in claim 1, wherein translating the text associated with the instance of the annotation comprises preserving a segment of the source document in the source language.

8. The method recited in claim 1, wherein translating the text associated with the instance of the annotation comprises preserving a number of spaces around the translated text associated with the annotation in the target document.

9. The method recited in claim 1, wherein translating the text associated with the instance of the annotation comprises preserving the annotation in the target document.

10. The method recited in claim 1, wherein the annotation comprises an extensible markup language tag.

11. The method recited in claim 1, wherein the annotation is associated with the source language.

12. The method recited in claim 1, wherein the annotation is associated with the target language.

13. The method recited in claim 1, further comprising training the translation engine to identify the annotation in the source document using annotated entries in the translation memory database.

14. The method recited in claim 1, wherein the associated annotated segment of text is adjacent the annotation.

15. The method recited in claim 1, wherein the translating of the identified strings using the translation engine trained using the translation memory is performed without reference to the annotation.

16. The method recited in claim 1, wherein the translation engine is further trained with the output context associated with the annotation.

17. The method recited in claim 1, wherein the output context comprises one or more words in the target document proximate to the annotation.

18. The method recited in claim 1, wherein the output context comprises a defined semantic behavior for at least one of a date, a number and a product name.

19. A system for translating a document, comprising:

defined translation rules associated with annotations occurring in a translation memory database, wherein the translation rules comprise an output context;

a communications interface configured to receive a source document in a source language, the source document comprising a string and an annotated segment of text, the annotated segment of text comprising text and an instance of an annotation;

a statistical machine translation engine trained for statistical machine translation using both parallel segments, the annotation integrated into the parallel segments, and a translation memory comprising the annotation, to modify statistical probability values used in the statistical machine translation engine, the modified statistical probability values stored for use in the statistical machine translation by the statistical machine translation engine;

a translation memory database including translation rules integrated into the statistical machine translation used by the statistical machine translation engine, the integration including training the statistical translation engine to recognize the annotations in the translation memory while performing the statistical machine translation; and

a processor configured to translate the source document as a whole using the statistical machine translation engine without breaking the source document into segments that are translated according to whether a match appears in the translation memory database, the statistical machine translation engine configured to;

identify the annotations and the associated annotated segments of the text in the source document to be processed using the translation rules, and the strings in the source document to be translated using statistical machine translation,

translate the identified strings in the source document using statistical machine translation,

process the identified instances of the annotations and the associated annotated segments of text according to the translation rules in the translation memory database, at least one of the translation rules being associated with the annotation, and

generate a target document in a target language based on the translated string and the translated text associated with the annotation.

20. The system recited in claim 19, wherein the annotation indicates a word class.

21. The system recited in claim 20, wherein the word class is associated with a value.

22. The system recited in claim 20, wherein the output context is associated with the word class.

23. The system recited in claim 19, wherein the annotation indicates a boundary.

24. The system recited in claim 23, wherein the processor is configured to translate text occurring after the instance of the annotation separately from text occurring before the instance of the annotation.

25. The system recited in claim 19, wherein the processor is configured to preserve a segment of the source document in the source language.

26. The system recited in claim 19, wherein the processor is configured to preserve a number of spaces around the text associated with the annotation in the target document.

27. The system recited in claim 19, wherein the processor is configured to preserve the annotation in the target document.

28. The system recited in claim 19, wherein the annotation comprises an extensible markup language tag.

29. The system recited in claim 19, wherein the annotation is associated with the source language.

30. The system recited in claim 19, wherein the annotation is associated with the target language.

31. The system recited in claim 19, wherein the processor is further configured to train the statistical machine translation engine to identify the annotation in the source document using annotated entries in the translation memory database.

32. The system recited in claim 19, wherein the processor is configured to translate the identified string using the statistical machine translation engine without reference to the annotation.

33. A non-transitory computer readable storage medium having embodied thereon a computer program having instructions executable by a computer to perform a method for translating a document, the method comprising the steps:

defining translation rules associated with annotations occurring in a translation memory database, wherein the translation rules comprise an output context;

integrating the translation rules in the translation memory database into statistical machine translation used by a translation engine, the integration including training the translation engine to recognize annotations in the translation memory database while performing statistical machine translation;

training the statistical machine translation of the translation engine using both the translation memory and parallel segments to modify statistical probability values;

storing the statistical probability values modified during the training, the stored values for use in the statistical machine translation by the translation engine;

receiving a source document in a source language, the source document comprising a string and an annotated segment of text, the annotated segment of text comprising text and an instance of an annotation;

translating the source document as a whole using the translation engine without breaking the source document into segments that are translated according to whether a match appears in the translation memory database, the translation engine configured for;

identifying annotations and associated annotated segments of the text in the source document to be processed using the translation rules, and strings in the source document to be translated using the statistical machine translation,

translating the identified strings in the source document using the statistical machine translation, and

processing the identified instances of the annotations and the associated annotated segments of text according to the translation rules using the translation memory database, at least one of the translation rules being associated with the annotation; and

generating a target document in a target language based on the translated string and the translated text associated with the annotation.

34. The non-transitory computer readable storage medium recited in claim 33, wherein the annotation indicates a word class.

35. The non-transitory computer readable storage medium recited in claim 34, wherein the word class is associated with a value.

36. The non-transitory computer readable storage medium recited in claim 34, wherein the output context is associated with the word class.

37. The non-transitory computer readable storage medium recited in claim 33, wherein the annotation indicates a boundary.

38. The non-transitory computer readable storage medium recited in claim 37, further comprising an instruction executable by the computer for translating text occurring after the instance of the annotation separately from text occurring before the instance of the annotation.

39. The non-transitory computer readable storage medium recited in claim 33, further comprising an instruction executable by the computer for preserving a segment of the source document in the source language.

40. The non-transitory computer readable storage medium recited in claim 33, further comprising an instruction for preserving a number of spaces around the translated text associated with the annotation in the target document.

41. The non-transitory computer readable storage medium recited in claim 33, further comprising an instruction for preserving the annotation in the target document.

42. The non-transitory computer readable storage medium recited in claim 33, wherein the annotation comprises an extensible markup language tag.

43. The non-transitory computer readable storage medium recited in claim 33, wherein the annotation is associated with the source language.

44. The non-transitory computer readable storage medium recited in claim 33, wherein the annotation is associated with the target language.

45. The non-transitory computer readable storage medium recited in claim 33, further comprising an instruction executable by the computer training the translation engine to identify the annotation in the source document using annotated entries in the translation memory database.

46. The non-transitory computer readable storage medium recited in claim 33, wherein the translating of the identified string using the translation engine trained using the translation memory is performed without reference to the annotation.