Using game responses to gather data

• US 8,606,745 B1
• Filed: 10/03/2011
• Issued: 12/10/2013
• Est. Priority Date: 12/03/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Haveliwala, Taher, Gomes, Benedict, Singhal, Amitabh K
• Primary Examiner(s): Choi, Yuk Ting
• Application Number:
  US13/252,099
• Time in Patent Office: 799 Days
• Field of Search: None
• US Class Current: 707/602
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

A system provides images or questions to multiple game participants and receives labels or answers in response thereto. The system uses the labels or answers for various data gathering purposes.

Claims

1. A computer-implemented method comprising:

providing, by one or more processors and to a user, a plurality of questions with known answers;

receiving, at the one or more processors and from the user, answers to the plurality of questions with known answers;

determining, by the one or more processors, a quantity of the answers that are correct answers;

determining, by the one or more processors and based on the quantity of answers that are correct answers, that the user is human;

providing, by the one or more processors, a first image to multiple users,

the multiple users including the user;

receiving, at the one or more processors and from the multiple users, terms identifying the first image;

discarding, by the one or more processors, terms, of the received terms, that were received from users, of the multiple users, determined not to be human,

the discarding forming a set of terms;

identifying, by the one or more processors and from the set of terms, a first term and a second term as related terms; and

associating, by the one or more processors, the first term as a label with the first image.

2. The method of claim 1, further comprising:

identifying the related terms as synonyms.

3. The method of claim 1, further comprising:

supplying the first term as a suggestion for a third term of a user search query that is received after associating the first term as a label with the first image.

4. The method of claim 1, further comprising:

identifying the second term as excluded from being associated with the first image based on the second term matching a term in a list of excluded words.

5. The method of claim 4, further comprising:

providing an error message to a particular user, of the multiple users, from which the excluded second term is received.

6. The method of claim 1, where the first term is in a first language and the second term is in a second, different language, and where the method further comprises:

receiving a user search query that includes the first term; and

translating the user search query from the first language into the second, different language by replacing the first term, in the user search query, with the second term.

7. The method of claim 1, further comprising:

determining a number of users, of the multiple users, that provide the first term as identifying the first image;

comparing the number to a threshold value; and

determining that the label correctly identifies the first image based on the comparing.

8. The method of claim 7, further comprising:

assigning a score to a user, of the number of users, based on the label correctly identifying the first image; and

weighting, based on the score, subsequent terms received from the user.

9. The method of claim 7, further comprising:

identifying at least one of the multiple users as human based on the at least one of the multiple users providing the label that correctly identifies the first image.

10. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions, which, when executed by at least one processor, cause the at least one processor to;

provide, to a user, a plurality of questions with known answers;

receive, from the user, answers to the plurality of questions with known answers;

determine a quantity of the answers that are correct answers;

assign a score to the users based on the quantity of answers that are correct answers;

determine, based on the assigned score, that the user is human;

provide a first image to multiple users,

the multiple users including the user;

receive, from the multiple users, terms identifying the first image;

discard terms, of the received terms, that were not received from users, of the multiple users, determined not to be human,

the discarding forming a set of terms;

identify, from the set of terms, a first term and a second term as related terms; and

associate the first term as a label with the first image.

11. The non-transitory computer-readable medium of claim 10, where the instructions further comprise:

one or more instructions, which, when executed by the at least one processor, cause the at least one processor to;

identify the related terms as synonyms.

12. The non-transitory computer-readable medium of claim 10, where the instructions further comprise:

one or more instructions, which, when executed by the at least one processor, cause the at least one processor to;

supply the first term as a suggestion for a third term of a user search query received after associating the first term as a label with the first image.

13. The non-transitory computer-readable medium of claim 10, where the instructions further comprise:

one or more instructions, which, when executed by the at least one processor, cause the at least one processor to;

identify the second term as excluded from being associated with the first image based on the second term matching a term in a list of excluded words.

14. The non-transitory computer-readable medium of claim 13, where the instructions further comprise:

one or more instructions, which, when executed by the at least one processor, cause the at least one processor to;

provide an error message to a particular user, of the multiple users, from which the excluded second term is received.

15. The non-transitory computer-readable medium of claim 10, where the first term is in a first language and the second term is in a second, different language, and where the instructions further comprise:

one or more instructions, which, when executed by the at least one processor, cause the at least one processor to;

translate a user search query, which includes the first term, from the first language into the second, different language by replacing the first term in the user search query with the second term.

16. The non-transitory computer-readable medium of claim 10, where the instructions further comprise:

one or more instructions, which, when executed by the at least one processor, cause the at least one processor to;

determine a number of users, of the multiple users, that provide the first term as identifying the first image;

compare the number to a threshold value; and

determine that the label correctly identifies the first image based on the comparing.

17. The non-transitory computer-readable medium of claim 16, where the score assigned to the user is further based on the label correctly identifying the first image, and where the instructions further comprise:

one or more instructions, which, when executed by the at least one processor, cause the at least one processor to;

weight, based on the score, subsequent terms received from the user.

18. The non-transitory computer-readable medium of claim 16, where the instructions further comprise:

one or more instructions, which, when executed by the at least one processor, cause the at least one processor to;

identify at least one of the multiple users as human based on the at least one of the multiple users providing the label that correctly identifies the first image.

19. A system comprising:

a server device to;

provide, to a user, a plurality of questions with known answers;

receive, from the user, answers to the plurality of questions with known answers;

determine a quantity of the answers that are correct answers;

assign a score to the users based on the quantity of answers that are correct answers;

determine, based on the assigned score, that the user is human;

provide a first image to multiple users,

the multiple users including the user;

receive, from the multiple users, terms identifying the first image;

discard terms, of the received terms, that were received from users, of the multiple users, determined not to be human,

the discarding forming a set of terms;

identify, from the set of terms, a first term and a second term as related terms; and

associate the first term as a label with the first image.

20. The system of claim 19, where the server device further is to:

identify the related terms as synonyms.

21. The system of claim 19, where the server device further is to:

supply the first term as a suggestion for a third term of a user search query that is received after associating the first term as a label with the first image.

22. The system of claim 19, where the server device further is to:

identify the second term as excluded from being associated with the first image based on the second term matching a term in a list of excluded words.

23. The system of claim 22, where the server device further is to:

provide an error message to a particular user, of the multiple users, from which the excluded second term is received.

24. The system of claim 19, where the first term is in a first language and the second term is in a second, different language, and where the server device further is to:

translate a user search query, which includes the first term, from the first language into the second, different language by replacing the first term, in the user search query, with the second term.

25. The system of claim 19, where the server device further is to:

determine a number of users, of the multiple users, that provide the first term as identifying the first image;

compare the number to a threshold value; and

determine that the label correctly identifies the first image based on the comparing.

26. The system of claim 25, where the score assigned to the user is further based on the label correctly identifying the first image, and where the server device is further to:

weight, based on the score, subsequent terms received from the user.

27. The system of claim 25, where the server device further is to:

identify at least one of the multiple users as human based on the at least one of the multiple users providing the label that correctly identifies the first image.