Using a human computation game to improve search engine performance

• US 8,285,706 B2
• Filed: 06/10/2009
• Issued: 10/09/2012
• Est. Priority Date: 06/10/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chandrasekar, Raman, Quirk, Christopher Brian, Shah, Sarthak Deepak, Richardson, Matthew, Gupta, Abhishek, Ma, Hao, Champness Burges, Christopher John
• Primary Examiner(s): Ehichioya, Fred I
• Assistant Examiner(s): SMITH, GARRETT A
• Application Number:
  US12/481,595
• Publication Number:
  US 20100317444A1
• Time in Patent Office: 1,217 Days
• Field of Search: None
• US Class Current: 707/711
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Human computation games are provided wherein a player is shown a page, such as a web page. The player is then asked to provide one or more terms that are intended to cause a search engine to return the page in response to performing a query using the terms. The terms provided by the player during game play are then collected, stored, and utilized to improve the performance of the search engine.

Claims

1. A computer-implemented method for using a human computation game to improve search engine performance, the computer-implemented method comprising performing computer-implemented operations for:

displaying a page to two or more players via a multi-player game;

upon displaying the page,

receiving first terms from a first player and receiving second terms from a second player,

causing a search engine to return first candidate pages in response to performing a first query using the first terms,

causing the search engine to return second candidate pages in response to performing a second query using the second terms,

assigning points to the first player if the first candidate pages returned by the search engine include the page displayed before the second candidate pages returned by the search engine include the page displayed, and

assigning the points to the second player if the second candidate pages returned by the search engine include the page displayed before the first candidate pages returned by the search engine include the page displayed;

storing one or more of the first terms and the second terms provided by the two or more players during play of the multi-player game and associated data; and

utilizing the one or more of the first terms and the second terms and the associated data to improve results returned by the search engine.

2. The computer-implemented method of claim 1, further comprising:

displaying a second page via a single-player game;

upon displaying the second page,

receiving one or more terms from a player via the single-player game,

causing the search engine to return the candidate pages in response to performing a query using the terms, and

assigning second points to the player if the second page displayed to the player is included in the candidate pages returned by the search engine;

storing the terms provided by the player during play of the single-player game and associated second data; and

utilizing the terms and the associated second data to improve results returned by the search engine.

3. The computer-implemented method of claim 1, further comprising:

displaying a third page to the two or more players via a second multi-player game;

upon displaying the third page,

receiving third terms from the first player and receiving fourth terms from the second player,

causing the search engine to return third candidate pages in response to performing a third query using the third terms,

causing the search engine to return fourth candidate pages in response to performing a fourth query using the fourth terms, and

assigning third points to the first player and the second player when the first player and the second player agree and correctly indicate that they are viewing the same or a different page as the other player;

storing one or more of the third terms and the fourth terms provided by the two or more players during play of the second multi-player game and associated third data; and

utilizing the one or more of the third terms and the fourth terms and the associated third data to improve results returned by the search engine.

4. The computer-implemented method of claim 1, wherein utilizing the one or more of the first terms and the second terms to improve results returned by the search engine comprises utilizing the one or more of the first terms and the second terms to identify query alterations for use by the search engine.

5. The computer-implemented method of claim 1, wherein utilizing the one or more of the first terms and the second terms to improve results returned by the search engine comprises utilizing the one or more of the first terms and the second terms to provide metadata for the page.

6. The computer-implemented method of claim 1, wherein utilizing the one or more of the first terms and the second terms to improve results returned by the search engine comprises utilizing the one or more of the first terms and the second terms to identify query structure.

7. The computer-implemented method of claim 1, wherein utilizing the one or more of the first terms and the second terms to improve results returned by the search engine comprises performing a cognitive analysis on the one or more of the first terms and the second terms to identify searching behavior.

8. The computer-implemented method of claim 1, wherein utilizing the one or more of the first terms and the second terms to improve results returned by the search engine comprises utilizing the one or more of the first terms and the second terms determine where on the page the first player'"'"'s or the second player'"'"'s eyes are focused when providing the one or more of the first terms and the second terms.

9. The computer-implemented method of claim 1, wherein utilizing the one or more of the first terms and the second terms to improve results returned by the search engine comprises utilizing the one or more of the first terms and the second terms to identify a ranking issue with the page.

10. A computer-readable storage medium having computer-readable instructions stored thereupon which, when executed by a computer, cause the computer to:

display a page to two or more players via a multi-player game;

when the page is displayed,

receive first terms from a first player and receiving second terms from a second player,

cause a search engine to return first candidate pages in response to performing a first query using the first terms,

cause the search engine to return second candidate pages in response to performing a second query using the second terms, and

assign points to the first player and the second player when the first player and the second player agree and correctly indicate that they are viewing the same or a different page as the other player;

store one or more of the first terms and the second terms provided by the two or more players during play of the multi-player game and associated data; and

utilize the one or more of the first terms and the second terms and the associated data to improve results returned by the search engine.

11. The computer-readable storage medium of claim 10, having further computer-readable instructions stored thereupon which, when executed by the computer, cause the computer to:

display a second page via a single-player game;

when the second page is displayed,

receive one or more terms from a player via the single-player game,

cause the search engine to return the candidate pages in response to performing a query using the terms, and

assign second points to the player if the second page displayed to the player is included in the candidate pages returned by the search engine;

store the terms provided by the player during play of the single-player game and associated second data; and

utilize the terms and the associated second data to improve results returned by the search engine.

12. The computer-readable storage medium of claim 10, having further computer-readable instructions stored thereupon which, when executed by the computer, cause the computer to:

display a third page to the two or more players via a second multi-player game;

when the third page is displayed,

receive third terms from the first player and receiving fourth terms from the second player,

cause the search engine to return third candidate pages in response to performing a third query using the third terms,

cause the search engine to return fourth candidate pages in response to performing a fourth query using the fourth terms,

assign points to the first player if the third candidate pages returned by the search engine include the third page displayed before the fourth candidate pages returned by the search engine include the third page displayed, and

assign the points to the second player if the fourth candidate pages returned by the search engine include the third page displayed before the third candidate pages returned by the search engine include the third page displayed;

store one or more of the third terms and the fourth terms provided by the two or more players during play of the second multi-player game and associated third data; and

utilizing the one or more of the third terms and the fourth terms and the associated third data to improve results returned by the search engine.

13. The computer-readable storage medium of claim 10, wherein to utilize the one or more of the first terms and the second terms to improve results returned by the search engine, the computer-readable instructions further cause the computer to utilize the one or more of the first terms and the second terms to identify query alterations for use by the search engine.

14. The computer-readable storage medium of claim 10, wherein to utilize the one or more of the first terms and the second terms to improve results returned by the search engine, the computer-readable instructions further cause the computer to utilize the one or more of the first terms and the second terms to provide metadata for the page.

15. The computer-readable storage medium of claim 10, wherein to utilize the one or more of the first terms and the second terms to improve results returned by the search engine, the computer-readable instructions further cause the computer to utilize the one or more of the first terms and the second terms to identify query structure.

16. The computer-readable storage medium of claim 10, wherein to utilize the one or more of the first terms and the second terms to improve results returned by the search engine, the computer-readable instructions further cause the computer to perform a cognitive analysis on the one or more of the first terms and the second terms to identify searching behavior.

17. The computer-readable storage medium of claim 10, wherein to utilize the one or more of the first terms and the second terms to improve results returned by the search engine, the computer-readable instructions further cause the computer to utilize the one or more of the first terms and the second terms determine where on the page the first player'"'"'s or the second player'"'"'s eyes are focused when providing the one or more of the first terms and the second terms.

18. The computer-readable storage medium of claim 10, wherein to utilize the one or more of the first terms and the second terms to improve results returned by the search engine, the computer-readable instructions further cause the computer to utilize the one or more of the first terms and the second terms to identify a ranking issue with the page.

19. A computer-implemented method for using a human computation game to improve search engine performance, the computer-implemented method comprising performing computer-implemented operations for:

displaying a page to two or more players via a multi-player game;

upon displaying the page,

receiving first terms from a first player and receiving second terms from a second player,

causing a search engine to return first candidate pages in response to performing a first query using the first terms,

causing the search engine to return second candidate pages in response to performing a second query using the second terms, and

assigning points to the first player and the second player when the first player and the second player agree and correctly indicate that they are viewing the same or a different page as the other player;

storing one or more of the first terms and the second terms provided by the two or more players during play of the multi-player game and associated data; and

utilizing the one or more of the first terms and the second terms and the associated data to improve results returned by the search engine.

20. The computer-implemented method of claim 19, further comprising:

displaying a second page via a single-player game;

upon displaying the second page,

receiving one or more terms from a player via the single-player game,

causing the search engine to return the candidate pages in response to performing a query using the terms, and

assigning second points to the player if the second page displayed to the player is included in the candidate pages returned by the search engine;

storing the terms provided by the player during play of the single-player game and associated second data; and

utilizing the terms and the associated second data to improve results returned by the search engine.