Systems and methods for correcting translations in multi-user multi-lingual communications

• US 9,245,278 B2
• Filed: 03/27/2015
• Issued: 01/26/2016
• Est. Priority Date: 02/08/2013
• Status: Active Grant

• Current Assignee: Mz Ip Holdings, LLC (AppLovin Corporation)
• Original Assignee: Machine Zone, Inc. (AppLovin Corporation)
• Inventors: Orsini, Francois, Bojja, Nikhil, Nedunchezhian, Arun
• Primary Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US14/671,243
• Publication Number:
  US 20150262209A1
• Time in Patent Office: 305 Days
• Field of Search: 704 1- 10
• US Class Current: 1/1
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/0208 Trade or exchange of goods or services in exchange for incentives or rewards
• CPC Class Codes: G06Q 30/0217 involving input on products or services in exchange for incentives or rewards
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]

Abstract

Various embodiments described herein facilitate multi-lingual communications. The systems and methods of some embodiments enable multi-lingual communications through different modes of communication including, for example, Internet-based chat, e-mail, text-based mobile phone communications, postings to online forums, postings to online social media services, and the like. Certain embodiments implement communication systems and methods that translate text between two or more languages. Users of the systems and methods may be incentivized to submit corrections for inaccurate or erroneous translations, and may receive a reward for these submissions. Systems and methods for assessing the accuracy of translations are described.

Claims

1. A method comprising:

performing by one or more computer processors;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a comparison of a word-based feature of the original text and the translation correction, wherein calculating the at least one metric comprises determining a difference in a number of words, characters, emojis, numbers, or punctuation marks between the original text and the translation correction; and

determining an accuracy of the translation correction based on the at least one metric.

2. The method of claim 1, further comprising updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

3. The method of claim 2, further comprising revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

4. The method of claim 1, further comprising:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

5. A method comprising:

performing by one or more computer processors;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a comparison of a word-based feature of the original text and the translation correction, wherein calculating the metric comprises;

identifying a longest common subsequence of characters that occurs in the original text and the translation correction wherein each character in the subsequence is a number or a punctuation mark; and

calculating the metric as a quotient of a length of the subsequence and a maximum of a

length of the original text and a length of the translation correction; and

determining an accuracy of the translation correction based on the at least one metric.

6. The method of claim 5, further comprising updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

7. The method of claim 6, further comprising revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

8. The method of claim 5, further comprising:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

9. A method comprising:

performing by one or more computer processors;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a comparison of a language-based feature of the original text and the translation correction, wherein the at least one metric is based on the comparison of the language-based feature, and wherein calculating the at least one metric comprises determining a difference in a number of occurrences of a part of speech in the original text and in the translation correction; and

determining an accuracy of the translation correction based on the at least one metric.

10. The method of claim 9, wherein the part of speech is a verb, an adjective, an adverb, a noun, or a proper noun, and wherein the verb part of speech comprises a verb form selected from the group consisting of past tense, gerund, past participle, non-third person singular present, and third person singular present.

11. The method of claim 9, further comprising updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

12. The method of claim 11, further comprising revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

13. The method of claim 9, further comprising:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

14. A method comprising:

performing by one or more computer processors;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a comparison of a word-alignment feature of the original text and the translation correction, wherein calculating the at least one metric comprises determining an alignment of at least one word in the translation correction and a corresponding at least one word in the original message; and

determining an accuracy of the translation correction based on the at least one metric.

15. The method of claim 14, further comprising updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

16. The method of claim 15, further comprising revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

17. The method of claim 14, further comprising:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

18. A method comprising:

performing by one or more computer processors;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a grammar-based feature of the translation correction, wherein calculating the at least one metric comprises;

generating a part-of-speech n-gram representation of the translation correction; and

computing a probability of the n-gram representation for the second language; and

determining an accuracy of the translation correction based on the at least one metric.

19. The method of claim 18, further comprising updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

20. The method of claim 19, further comprising revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

21. The method of claim 18, further comprising:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

22. A method comprising:

performing by one or more computer processors;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based a comparison of a word-based feature of the original text and the translation correction and a comparison of a language-based feature of the original text and the translation correction;

determining an accuracy of the translation correction based on the at least one metric, wherein determining the accuracy of the translation correction comprises;

multiplying each of the first metric and the second metric by a respective coefficient; and

determining the accuracy of the translation based on whether a combination of the multiplied metrics exceeds a first threshold.

23. The method of claim 22, further comprising updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

24. The method of claim 23, further comprising revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

25. The method of claim 22, further comprising:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

26. A method comprising:

performing by one or more computer processors;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on one or more of;

(a) a comparison of a word-based feature of the original text and the translation correction;

(b) a comparison of a language-based feature of the original text and the translation correction;

(c) a comparison of a word-alignment feature of the original text and the translation correction; and

(d) a grammar-based feature of the translation correction; and

determining an accuracy of the translation correction based on the at least one metric wherein the at least one metric further comprises a third metric based on the comparison of the word-alignment feature, and wherein determining the accuracy of the translation correction further comprises;

multiplying the third metric by a respective coefficient;

combining the multiplied third metric with the multiplied first metric and the multiplied second metric; and

comparing the combination of multiplied metrics with a second threshold.

27. The method of claim 26, further comprising updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

28. The method of claim 27, further comprising revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

29. The method of claim 26, further comprising:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

30. A system comprising:

one or more computer processors programmed to perform operations comprising;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a comparison of a word-based feature of the original text and the translation correction, wherein calculating the at least one metric comprises determining a difference in a number of words, characters, emojis, numbers, or punctuation marks between the original text and the translation correction; and

determining an accuracy of the translation correction based on the at least one metric.

31. The system of claim 30 wherein the operations further comprise:

updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

32. The system of claim 31 wherein the operations further comprise:

revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

33. The system of claim 30 wherein the operations further comprise:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

34. A system comprising:

one or more computer processors programmed to perform operations comprising;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a comparison of a word-based feature of the original text and the translation correction, wherein calculating the metric comprises;

identifying a longest common subsequence of characters that occurs in the original text and the translation correction wherein each character in the subsequence is a number or a punctuation mark; and

calculating the metric as a quotient of a length of the subsequence and a maximum of a length of the original text and a length of the translation correction; and

determining an accuracy of the translation correction based on the at least one metric.

35. The system of claim 34 wherein the operations further comprise:

updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

36. The system of claim 35 wherein the operations further comprise:

revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

37. The system of claim 34 wherein the operations further comprise:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

38. A method comprising:

one or more computer processors programmed to perform operations comprising;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a comparison of a language-based feature of the original text and the translation correction, wherein the at least one metric is based on the comparison of the language-based feature, and wherein calculating the at least one metric comprises determining a difference in a number of occurrences of a part of speech in the original text and in the translation correction; and

determining an accuracy of the translation correction based on the at least one metric.

39. The system of claim 38 wherein the operations further comprise:

updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

40. The system of claim 39 wherein the operations further comprise:

revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

41. The system of claim 38 wherein the operations further comprise:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

42. The system of claim 38, wherein the part of speech is a verb, an adjective, an adverb, a noun, or a proper noun, and wherein the verb part of speech comprises a verb form selected from the group consisting of past tense, gerund, past participle, non-third person singular present, and third person singular present.

43. A system comprising:

one or more computer processors programmed to perform operations comprising;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a comparison of a word-alignment feature of the original text and the translation correction, wherein calculating the at least one metric comprises determining an alignment of at least one word in the translation correction and a corresponding at least one word in the original message; and

determining an accuracy of the translation correction based on the at least one metric.

44. The system of claim 43 wherein the operations further comprise:

updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

45. The system of claim 44 wherein the operations further comprise:

revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

46. The system of claim 43 wherein the operations further comprise:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

47. A system comprising:

one or more computer processors programmed to perform operations comprising;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on a grammar-based feature of the translation correction, wherein calculating the at least one metric comprises;

generating a part-of-speech n-gram representation of the translation correction; and

computing a probability of the n-gram representation for the second language; and

determining an accuracy of the translation correction based on the at least one metric.

48. The system of claim 47 wherein the operations further comprise:

updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

49. The system of claim 48 wherein the operations further comprise:

revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

50. The system of claim 47 wherein the operations further comprise:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

51. A system comprising:

one or more computer processors programmed to perform operations comprising;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based a comparison of a word-based feature of the original text and the translation correction and a comparison of a language-based feature of the original text and the translation correction;

determining an accuracy of the translation correction based on the at least one metric, wherein determining the accuracy of the translation correction comprises;

multiplying each of the first metric and the second metric by a respective coefficient; and

determining the accuracy of the translation based on whether a combination of the multiplied metrics exceeds a first threshold.

52. The system of claim 51 wherein the operations further comprise:

updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

53. The system of claim 52 wherein the operations further comprise:

revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

54. The system of claim 51 wherein the operations further comprise:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.

55. A system comprising:

one or more computer processors programmed to perform operations comprising;

obtaining an original text message in a first language authored by a first user;

obtaining an initial translation of the original text message in a second language;

obtaining a translation correction of the initial translation, wherein the translation correction is authored by a second user;

calculating at least one metric associated with the translation correction, the at least one metric being based on one or more of;

(a) a comparison of a word-based feature of the original text and the translation correction;

(b) a comparison of a language-based feature of the original text and the translation correction;

(c) a comparison of a word-alignment feature of the original text and the translation correction; and

(d) a grammar-based feature of the translation correction; and

determining an accuracy of the translation correction based on the at least one metric wherein the at least one metric further comprises a third metric based on the comparison of the word-alignment feature, and wherein determining the accuracy of the translation correction further comprises;

multiplying the third metric by a respective coefficient;

combining the multiplied third metric with the multiplied first metric and the multiplied second metric; and

comparing the combination of multiplied metrics with a second threshold.

56. The system of claim 55 wherein the operations further comprise:

updating a confidence score for the second user based on the accuracy of the translation correction, the confidence score representing a likelihood that the second user will provide an accurate translation of a text message at a later time.

57. The system of claim 56 wherein the operations further comprise:

revoking the second user'"'"'s translation privileges when the confidence score falls below a threshold value.

58. The system of claim 55 wherein the operations further comprise:

offering an incentive to the second user to provide the translation correction; and

rewarding the second user with the respective incentive when the translation correction is determined to be accurate.