Systems and methods for multi-user multi-lingual communications

• US 8,990,068 B2
• Filed: 06/03/2014
• Issued: 03/24/2015
• Est. Priority Date: 02/08/2013
• Status: Expired due to Fees

• Current Assignee: Mz Ip Holdings, LLC (AppLovin Corporation)
• Original Assignee: Machine Zone, Inc. (AppLovin Corporation)
• Inventors: Orsini, Francois, Bojja, Nikhil
• Primary Examiner(s): He, Jialong
• Application Number:
  US14/294,684
• Publication Number:
  US 20140288917A1
• Time in Patent Office: 294 Days
• Field of Search: 704/2, 704/4, 704/8
• US Class Current: 704/2
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G06Q 10/06398 Performance of employee with respect to a job function
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0217 involving input on products or services in exchange for incentives or rewards

Abstract

Various embodiments described herein facilitate multi-lingual communications. The systems and methods of some embodiments enable multi-lingual communications through different modes of communication including, for example, Internet-based chat, e-mail, text-based mobile phone communications, postings to online forums, postings to online social media services, and the like. Certain embodiments implement communication systems and methods that translate text between two or more languages. Users of the systems and methods may be incentivized to submit corrections for inaccurate or erroneous translations, and may receive a reward for these submissions. Systems and methods for assessing the accuracy of translations are described.

Claims

1. A method implemented by a data processing apparatus, the method comprising:

selecting a mixture of old training data and new training data, the old training data comprising an old text message for which a correct translation to a different language is known, the new training data comprising a new text message for which a correct translation to the different language is not known;

sending a plurality of respective requests at different times to a client device of a user, the requests comprising (i) a respective request for the user to translate at least one of the old training data and the new training data and (ii) a respective incentive for the translation;

after sending a particular request, receiving a translation from the client device for the old training data of the particular request;

comparing the received translation with the correct translation for the old training data;

determining an accuracy of the received translation based on the comparison;

detecting collusion between the user and a second user by identifying a pre-existing relationship between the user and the second user; and

updating a confidence score for the user based on the translation using item response theory to identify a deviation from a norm in user translation accuracy, the confidence score representing a likelihood that the user will provide an accurate translation of a text message to the different language at a later time.

2. The method of claim 1, wherein the user is a participant in an online game.

3. The method of claim 1, wherein the respective incentive comprises at least one of a virtual good and a virtual currency for an online game.

4. The method of claim 1, wherein determining an accuracy of the translation received from the user comprises at least one of (i) computing word error rate (WER) and (ii) using bilingual evaluation understudy (BLEU).

5. The method of claim 1, further comprising revoking the user'"'"'s translation privileges when the confidence score falls below a threshold value.

6. The method of claim 1, further comprising rewarding the user with the respective incentive when the user'"'"'s translation is determined to be correct.

7. The method of claim 1, wherein identifying the pre-existing relationship comprises analyzing a social network of at least one of the user and the second user.

8. A system comprising:

a computer readable medium having instructions stored thereon; and

a data processing apparatus configured to execute the instructions to perform operations comprising;

selecting a mixture of old training data and new training data, the old training data comprising an old text message for which a correct translation to a different language is known, the new training data comprising a new text message for which a correct translation to the different language is not known;

sending a plurality of respective requests at different times to a client device of a user, the requests comprising (i) a respective request for the user to translate at least one of the old training data and the new training data and (ii) a respective incentive for the translation;

after sending a particular request, receiving a translation from the client device for the old training data of the particular request;

comparing the received translation with the correct translation for the old training data;

determining an accuracy of the received translation based on the comparison;

detecting collusion between the user and a second user by identifying a pre-existing relationship between the user and the second user; and

updating a confidence score for the user based on the translation using item response theory to identify a deviation from a norm in user translation accuracy, the confidence score representing a likelihood that the user will provide an accurate translation of a text message to the different language at a later time.

9. The system of claim 8, wherein the user is a participant in an online game.

10. The system of claim 8, wherein the respective incentive comprises at least one of a virtual good and a virtual currency for an online game.

11. The system of claim 8, wherein determining an accuracy of the translation received from the user comprises at least one of (i) computing word error rate (WER) and (ii) using bilingual evaluation understudy (BLEU).

12. The system of claim 8, the operations further comprising revoking the user'"'"'s translation privileges when the confidence score falls below a threshold value.

13. The system of claim 8, the operations further comprising rewarding the user with the respective incentive when the user'"'"'s translation is determined to be correct.

14. The system of claim 8, wherein identifying the pre-existing relationship comprises analyzing a social network of at least one of the user and the second user.

15. A computer program product stored in one or more non-transitory storage media for controlling a processing mode of a data processing apparatus, the computer program product being executable by the data processing apparatus to cause the data processing apparatus to perform operations comprising:

selecting a mixture of old training data and new training data, the old training data comprising an old text message for which a correct translation to a different language is known, the new training data comprising a new text message for which a correct translation to the different language is not known;

sending a plurality of respective requests at different times to a client device of a user, the requests comprising (i) a respective request for the user to translate at least one of the old training data and the new training data and (ii) a respective incentive for the translation;

after sending a particular request, receiving a translation from the client device for the old training data of the particular request;

comparing the received translation with the correct translation for the old training data;

determining an accuracy of the received translation based on the comparison;

detecting collusion between the user and a second user by identifying a pre-existing relationship between the user and the second user; and

updating a confidence score for the user based on the translation using item response theory to identify a deviation from a norm in user translation accuracy, the confidence score representing a likelihood that the user will provide an accurate translation of a text message to the different language at a later time.

16. The computer program product of claim 15, wherein the user is a participant in an online game.

17. The computer program product of claim 15, wherein the respective incentive comprises at least one of a virtual good and a virtual currency for an online game.

18. The computer program product of claim 15, wherein determining an accuracy of the translation received from the user comprises at least one of (i) computing word error rate (WER) and (ii) using bilingual evaluation understudy (BLEU).

19. The computer program product of claim 15, the operations further comprising revoking the user'"'"'s translation privileges when the confidence score falls below a threshold value.

20. The computer program product of claim 15, the operations further comprising rewarding the user with the respective incentive when the user'"'"'s translation is determined to be correct.

21. The computer program product of claim 15, wherein identifying the pre-existing relationship comprises analyzing a social network of at least one of the user and the second user.