Systems and methods for incentivizing user feedback for translation processing

• US 9,665,571 B2
• Filed: 03/21/2016
• Issued: 05/30/2017
• Est. Priority Date: 02/08/2013
• Status: Expired due to Fees

• Current Assignee: Mz Ip Holdings, LLC (AppLovin Corporation)
• Original Assignee: Machine Zone, Inc. (AppLovin Corporation)
• Inventors: Leydon, Gabriel, Orsini, Francois, Bojja, Nikhil
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US15/076,249
• Publication Number:
  US 20160203129A1
• Time in Patent Office: 435 Days
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06Q 30/0217 involving input on products or services in exchange for incentives or rewards
• CPC Class Codes: G07F 17/3255 Incentive, loyalty and/or promotion schemes, e.g. comps, gaming associated with a purchase, gaming funded by advertisements

Abstract

Systems and methods of various embodiments may enable or refine translation of text between a first language and a second language. In particular, systems and methods may enable or refine a text translation by soliciting and/or receiving feedback for: translation of a first word or phrase from a first language to a second language; transformation of the first word or phrase (in the first language) to a second word or phrase in the first language; or transformation of the first word or phrase (in the first language) to a second word or phrase in the second language. The systems and methods of various embodiments may incentivize user feedback for failed translations in order to encourage user feedback, improve the quality of user feedback received, and to permit development of translation corpora that can evolve with time.

Claims

1. A computer-implemented method comprising:

performing by one or more computers;

selecting a word or phrase of a message that was not correctly translated from a first language to a second language;

selecting a plurality of users from whom to solicit user feedback for the translation, wherein each selected user has not submitted feedback more times than a quota for a time period;

sending a query requesting user assistance in translating the selected word or phrase to one or more of the plurality of users;

receiving at least one response to the query from one or more of the users to whom the query was sent;

determining that the at least one response is approved;

determining a credit based on a complexity of the selected word or phrase or an importance of the selected word or phrase;

crediting with the determined credit a respective account of one or more of the users who provided the at least one approved response;

updating at least one of a transformation module and a translation module according to the at least one approved response; and

using at least one computer processor and at least one of the updated transformation module and the updated translation module to translate a second message comprising the selected word or phrase.

2. The method of claim 1, further comprising:

selecting a second plurality of users from whom to solicit user feedback for the translation;

sending a second query requesting user assistance in translating the selected word or phrase to one or more of the second plurality of users;

receiving at least one second response to the second query from one or more of the second users to whom the second query was sent;

determining that the at least one second response is approved; and

crediting a respective account of one or more of the second users who provided the at least one approved second response.

4. The method of claim 1, further comprising:

receiving from a user, data that flags the translation as being incorrect.

5. The method of claim 1, wherein the at least one response comprises a definition for the word or phrase in the first language.

6. The method of claim 1, wherein the at least one response comprises a definition for the word or phrase in the second language.

7. The method of claim 1, wherein the query comprises a field configured to receive a text-based value.

8. The method of claim 1, wherein the query comprises a set of preselected definitions from which a user can choose a definition for the word or phrase.

9. The method of claim 8, wherein the at least one response comprises an identification of one of the preselected definitions.

10. The method of claim 8, wherein the set of preselected definitions comprises at least one definition provided by a user in response to a previous query.

11. The method of claim 1, further comprising:

evaluating a competency of one of the plurality of users based on an accuracy of the at least one response.

12. The method of claim 1, wherein updating the at least one of the transformation module and the translation module comprises:

storing a transformation or translation of the selected word or phrase based on the at least one approved response.

13. The method of claim 1, wherein determining that the at least one response is approved comprises:

determining that the at least one response matches another response received.

3. The method of claim wherein the determined credit comprises an in-game currency or an in-game item.

14. A computer-implemented system comprising:

one or more computers programmed to perform operations comprising;

selecting a word or phrase of a message that was not correctly translated from a first language to a second language;

selecting a plurality of users from whom to solicit user feedback for the translation, wherein each selected user has not submitted feedback more times than a quota for a time period;

sending a query requesting user assistance in translating the selected word or phrase to one or more of the plurality of users;

receiving at least one response to the query from one or more of the users to whom the query was sent;

determining that the at least one response is approved;

determining a credit based on a complexity of the selected word or phrase or an importance of the selected word or phrase;

crediting with the determined credit a respective account of one or more of the users who provided the at least one approved response;

updating at least one of a transformation module and a translation module according to the at least one approved response; and

using at least one computer processor and at least one of the updated transformation module and the updated translation module to translate a second message comprising the selected word or phrase.

15. The system of claim 14, wherein the determined credit comprises an in-game currency or an in-game item.

16. The system of claim 14, wherein the operations further comprise:

selecting a second plurality of users from whom to solicit user feedback for the translation;

sending a second query requesting user assistance in translating the selected word or phrase to one or more of the second plurality of users;

receiving at least one second response to the second query from one or more of the second users to whom the second query was sent;

determining that the at least one second response is approved; and

crediting a respective account of one or more of the second users who provided the at least one approved second response.

17. The system of claim 14, wherein determining that the at least one response is approved comprises:

determining that the at least one response matches another response received.

18. The system of claim 14, wherein the operations further comprise:

receiving, from a user, data that flags the translation as being incorrect.

19. The system of claim 14, wherein the at least one response comprises a definition for the word or phrase in the first language.

20. The system of claim 14, wherein the at least one response comprises a definition for the word or phrase in the second language.

21. The system of claim 14, wherein the query comprises a field configured to receive a text-based value.

22. The system of claim 14, wherein the query comprises a set of preselected definitions from which a user the person can choose a definition for the word or phrase.

23. The system of claim 22, wherein the at least one response comprises an identification of one of the preselected definitions.

24. The system of claim 22, wherein the set of preselected definitions comprises at least one definition provided by a user in response to a previous query.

25. The system of claim 14, wherein the operations further comprise:

evaluating a competency of one of the plurality of users based on an accuracy of the at least one response.

26. The system of claim 14, wherein updating the at least one of the transformation module and the translation module comprises:

storing a transformation or translation of the selected word or phrase based on the at least one approved response.

27. An article of manufacture comprising:

a non-transitory computer-readable storage medium having instructions stored thereon which, when executed by one or more computers, causes the computers to perform operations comprising;

selecting a word or phrase of a message that was not correctly translated from a first language to a second language;

selecting a plurality of users from whom to solicit user feedback for the translation;

wherein each selected user has not submitted feedback more times than a quota for a time period;

sending a query requesting user assistance in translating the selected word or phrase to one or more of the plurality of users;

receiving at least one response to the query from one or more of the users to whom the query was sent;

determining that the at least one response is approved;

determining a credit based on a complexity of the selected word or phrase or an importance of the selected word or phrase;

crediting with the determined credit a respective account of one or more of the users who provided the at least one approved response;

updating at least one of a transformation module and a translation module according to the at least one approved response; and

using at least one computer processor and at least one of the updated transformation module and the updated translation module to translate a second message comprising the selected word or phrase.