Trust scoring for language translation systems

• US 8,886,517 B2
• Filed: 06/29/2012
• Issued: 11/11/2014
• Est. Priority Date: 06/17/2005
• Status: Active Grant

• Current Assignee: SDL PLC (RWS Holdings Plc)
• Original Assignee: Language Weaver, Inc. (RWS Holdings Plc)
• Inventors: Soricut, Radu, Marcu, Daniel
• Primary Examiner(s): AZAD, ABUL K
• Application Number:
  US13/539,037
• Publication Number:
  US 20140006003A1
• Time in Patent Office: 865 Days
• Field of Search: 704 1- 10
• US Class Current: 704/2
• CPC Class Codes: G06F 40/51 Translation evaluation

Abstract

Systems and methods for generating trust scores for translations are described herein. According to some embodiments, methods for generating a trust score for a translation may include establishing a trust score for at least a portion of a first translation of a source text translated by a trusted translation system, the trust score representing an accuracy level for the first translation, comparing the first translation of the source text generated by the trusted translation system to a second translation of the source text generated by an untrusted translation system, and determining a trust score for the second translation based upon the comparison.

Claims

1. A method for generating a trust score for a translation, the method comprising:

establishing, using a processor, a trust score for at least a portion of a first translation of a source text translated by a trusted translation system, the trust score representing an accuracy level for the first translation;

comparing, using the processor, the first translation of the source text generated by the trusted translation system to a second translation of the source text generated by an untrusted translation system;

evaluating, using the processor, the source text using a language model to determine a translation complexity for the source text; and

determining, using the processor, a trust score for the second translation based upon the comparison and using the determined translation complexity.

2. The method according to claim 1, further comprising generating a confidence level for the trust score, the confidence level representing a likelihood that the trust score is accurate.

3. The method according to claim 2, further comprising modifying the trust score based upon the confidence level.

4. The method according to claim 2, wherein generating the confidence level for the trust score comprises:

comparing any of the source text, the second translation or combinations thereof to parallel corpora utilized to train the trusted translation system; and

determining the confidence level for the trust score based upon the comparison.

5. The method according to claim 1, wherein the untrusted translation system is a translation system that utilizes an at least partially unknown translation method.

6. The method according to claim 1, wherein the trust score for the first translation comprises an average of a sum of trust scores for translated target language segments of the source text.

7. A method for generating a trust score for a translation, the method comprising:

determining, using a processor, a translation complexity for a source text;

calculating, using the processor, a first trust score for a first translation of the source text performed by a trusted translation system, the trust score;

evaluating, using the processor, target language constructs included in a second translation of the source text generated by an untrusted translation system; and

comparing, using the processor, the first translation to the second translation;

calculating, using the processor, a second trust score for the second translation based upon any combination of the translation complexity, the target language constructs, the first trust score, and the comparison of the first translation to the second translation.

8. The method according to claim 7, evaluating a length of each of the target language constructs and determining a second trust score for each target language construct based upon the length.

9. The method according to claim 8, wherein generating the second trust score for the translation comprises calculating an average of a sum of trust scores for the target language constructs.

10. The method according to claim 7, further comprising verifying the second trust score for an untrusted translation by comparing a domain of the source text with a domain of the translation, wherein a divergence in domains indicates an inaccurate translation.

11. The method according to claim 7, wherein determining a translation complexity for the source text comprises evaluating source text provided to an untrusted translation system using a language model.

12. A system for generating a trust score for a translation, the system comprising:

a memory for storing executable instructions; and

a processor for executing the executable instructions, the executable instructions comprising;

a trust score generator that establishes a trust score for at least a portion of a first translation of a source text translated by a trusted translation system, the trust score representing an accuracy level for the first translation;

an analysis module that;

compares the first translation of the source text generated by the trusted translation system to a second translation of the source text generated by an untrusted translation system; and

evaluates the source text using a language model to determine a translation complexity for the source text; and

an inference module that infers a trust score for the second translation based upon the comparison and the translation complexity determined for the source text.

13. The system according to claim 12, further comprising a confidence level module that generates a confidence level for the trust score, the confidence level representing a likelihood that the trust score is accurate.

14. The system according to claim 13, wherein the confidence level module modifies the trust score based upon the confidence level.

15. The system according to claim 13, wherein the confidence level module generates the confidence level for the trust score by:

comparing any of the source text, the second translation or combinations thereof to parallel corpora utilized to train the trusted translation system; and

determining the confidence level for the trust score based upon the comparison.

16. The system according to claim 12, wherein the untrusted translation system is a translation system that utilizes an at least partially unknown translation method.

17. The system according to claim 12, wherein a trust score for the first translation comprises an average of a sum of trust scores for translated target language segments of the source text.

18. The system according to claim 12, wherein the trust score generator generates a ranked list of target translations based upon trust scores associated with the target translations, the target translations being generated by a plurality of untrusted target translation systems.