Machine translation using non-contiguous fragments of text

• US 20070150257A1
• Filed: 12/22/2005
• Published: 06/28/2007
• Est. Priority Date: 12/22/2005
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Cancedda, Nicola, Yamada, Kenji, Dymetman, Marc, Goutte, Cyril, Simard, Michel, Gaussier, Eric, Cavestro, Bruno
• Granted Patent: US 7,536,295 B2
• Time in Patent Office: Days
• US Class Current: 704/2
• CPC Class Codes: G06F 40/45 Example-based machine translation; Alignment

Abstract

A machine translation method for translating source text from a first language to target text in a second language includes receiving the source text in the first language and accessing a library of bi-fragments, each of the bi-fragments including a text fragment from the first language and a text fragment from the second language, at least some of the bi-fragments comprising non-contiguous bi-fragments in which at least one of the text fragment from the first language and the text fragment from the second language comprises a non-contiguous fragment.

Claims

1. A machine translation method for translating source text from a first:

language to target text in a second language, comprising;

receiving the source text in the first language;

accessing a library of bi-fragments, each of the bi-fragments including a text fragment from the first language and a text fragment from the second language, at least some of the bi-fragments comprising non-contiguous bi-fragments in which at least one of the text fragment from the first language and the text fragment from the second language comprises a non-contiguous fragment; and

retrieving text fragments from the second language from the library corresponding to text fragments in the source text.

2. The method of claim 1, wherein the retrieving of text fragments includes retrieving a non-contiguous bi-fragment for a text fragment in the first language for which the library includes a non-contiguous bi-fragment.

3. The method of claim 1, further comprising:

generating at least one target hypothesis, each of said target hypotheses comprising text fragments selected from the second language; and

evaluating the hypothesis with a translation scoring function.

4. The method of claim 2, wherein the generation of a hypothesis includes expanding an existing hypothesis, and where the existing hypothesis includes at least one non-contiguous bi-fragment which includes at least one gap, adding a text fragment to the existing hypothesis which covers at least one of the at least one gaps in the existing hypothesis.

5. The method of claim 2, wherein the generating of a target hypothesis includes performing a multiple stack beam-search.

6. The method of claim 2, wherein each word in the text is assigned a source position and wherein a hypothesis which would result in two bi-fragments covering the same source position in the first language is rejected.

7. The method of claim 2, wherein a hypothesis which positions a word in the second language in the same position as a word already there is rejected.

8. The method of claim 2, wherein the translation scoring function comprises at least one feature function which accounts for non-contiguous text fragments.

9. The method of claim 8, wherein the feature function comprises at least one of:

a first feature function which models bi-fragment discontinuities within known translations; and

a second feature function models bi-fragment reorderings whereby statistically probable reorderings are favored over improbable ones.

10. The method of claim 9, wherein the feature function models bi-fragment discontinuities by counting the number of gaps within the source and target fragments that make up the translation hypothesis.

11. The method of claim 1, wherein each non-contiguous bi-fragment in the library includes at least one gap in at least one of the text fragment from the first language and the text fragment from the second language.

12. The method of claim 11, further including, rejecting a target hypothesis which includes a gap and wherein there is no bi-fragment in the library which includes a contiguous or non-contiguous text fragment in the second language corresponding to the text fragment from the first language which covers the gap.

13. The machine translation method of claim 1, further comprising:

generating the library of bi-fragments including storing non-contiguous bi-fragments in memory.

14. The machine translation method of claim 2, wherein the at least one target hypothesis is a partial hypothesis which covers fewer than all words in the source text.

15. A machine translation system comprising:

a memory which stores a library of bi-fragments, each of the bi-fragments including a source text fragment from a first language and a corresponding target text fragment from the second language, at least some of the bi-fragments in the library comprising non-contiguous bi-fragments; and

a processing component which receives text comprising source text fragments selected from the first language and accesses the memory to retrieve stored text fragments from the second language including non-contiguous bi-fragments.

16. The machine translation system of claim 15, wherein the processing component comprises a translation scoring function which includes at least one of:

a first feature function which evaluates bi-fragment discontinuities; and

a second feature function which evaluates bi-fragment reorderings.

17. The machine translation system of claim 16, wherein the processing component further comprises:

instructions for generating at least one target hypothesis, each of said target hypotheses comprising text fragments selected from the second language; and

instructions for evaluating the hypothesis with a translation scoring function.

18. The machine translation system of claim 17, wherein the first feature function counts gaps in the target text of the hypothesis and corresponding source text.

19. The machine translation system of claim 17, wherein the second feature function measures a difference in order between source and target fragments by summing over each bi-fragment used to translate the source text, a number of words from the first language which are positioned to the left of the source fragment in the bi-fragment, which are translated by corresponding words in the target text that begin to the right of the target fragment in the bi-fragment.

20. The machine translation system of claim 16, wherein the translation scoring function combines the outputs of the first and second scoring functions to provide a score and wherein the processing component prunes hypotheses based on their scores.

21. The machine translation system of claim 16, wherein the non-contiguous bi-fragments each include at least one of a non-contiguous source text fragment and a non-contiguous target text fragment.

22. The machine translation system of claim 16, wherein the library associates each of a plurality of non-contiguous bi-fragments with a corresponding maximum gap in at least one of the source text fragment and the target text fragment.

23. An article of manufacture comprising a program storage medium readable by a computer comprising:

a processing component which translates source text comprising source text fragments selected from a first language and outputs at least one hypothesis comprising target text fragments selected from a second language, the processing component configured for accessing an associated memory to retrieve stored text fragments from the second language, the memory storing bi-fragments comprising a text fragment in the first language and a corresponding text fragment in the second language, the processing component including a scoring function which comprises;

a first feature function which evaluates bi-fragment discontinuities, and

a second feature function which evaluates bi-fragment reorderings,

whereby a hypothesis which includes a non-contiguous fragment in at least one of the first language and the second language is scored.