Modifying search result ranking based on corpus search statistics

• US 8,359,309 B1
• Filed: 02/07/2011
• Issued: 01/22/2013
• Est. Priority Date: 05/23/2007
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vasserman, Alexander L., Provine, John R., Basile, Claudio
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): Mamillapalli, Pavan
• Application Number:
  US13/022,366
• Time in Patent Office: 715 Days
• Field of Search: 707/748, 707/725, 707/749, 707/750, 707/727, 707/721
• US Class Current: 707/721
• CPC Class Codes: G06F 16/3325 Reformulation based on results of preceding query
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

Methods, systems, and apparatus, including computer program products, for ranking search results of a search query using corpus search statistics. In one aspect, a method includes determining a first relevance of a first corpus to a search query, determining a second relevance of a second corpus to the search query, determining a measure of relative relevance of the first corpus and the second corpus to the search query, and providing the measure of relative relevance to a ranking engine for ranking of search results for a new search corresponding to the search query.

Claims

1. A method implemented by data processing apparatus, the method comprising:

determining, for a plurality of search results responsive to a query, a respective count of times search results in the plurality of search results that refer to documents in a base corpus have been presented, and a respective count of times search results in the plurality of search results that refer to documents in the base corpus have been selected, wherein the respective counts for presentations and selections of search results that refer to documents in the base corpus are for searches initiated by users in a plurality of different countries who employ a specific language;

determining, for the plurality of search results responsive to the query, a respective count of times search results in the plurality of search results that refer to documents in a second corpus have been presented, and a respective count of times search results in the plurality of search results that refer to documents in the second corpus have been selected, wherein the respective counts for presentations and selections of search results that refer to documents in the second corpus are for searches initiated by users in the plurality of different countries who employ the specific language;

calculating a click through rate of the base corpus for the query based at least in part on the respective counts for the base corpus;

calculating a click through rate of the second corpus for the query based at least in part on the respective counts for the second corpus;

calculating a measure of relative relevance based at least in part on a ratio of the second corpus click through rate to the base corpus click through rate; and

providing the measure of relative relevance to a ranking engine for ranking of search results for a search corresponding to the query; and

wherein fewer search results of the plurality refer to documents in the second corpus than to documents in the base corpus.

2. The method of claim 1 wherein the respective counts for the base and second corpora are derived from language-specific data or country-specific data, and wherein calculating a click through rate for one of the corpora further comprises selecting which data to use for the respective counts based on a determination that the selected data will provide a reliable statistic.

3. The method of claim 1 wherein the base corpus comprises web pages and wherein the second corpus comprises videos.

4. The method of claim 1 wherein the base corpus comprises web pages and wherein the second corpus comprises images.

5. The method of claim 1 wherein the click through rate of the base corpus or of the second corpus is a ratio of the count of selections to the count of presentations.

6. The method of claim 1 wherein the ranking engine is configured to increase a score of a search result for the query that refers to a document in the second corpus based at least in part on the measure of relative relevance.

7. The method of claim 1 wherein the respective counts for the base and second corpora are derived from language-specific data and country-specific data, and wherein calculating a click through rate for one of the corpora is based on a weighted combination of the language-specific data and the country-specific data.

8. The method of claim 7 wherein an amount of the language-specific data is greater than an amount of the country-specific data.

9. A system comprising:

a storage device encoded with instructions; and

data processing apparatus operable to execute the instructions to perform operations comprising;

determining, for a plurality of search results responsive to a query, a respective count of times search results in the plurality of search results that refer to documents in a base corpus have been presented, and a respective count of times search results in the plurality of search results that refer to documents in the base corpus have been selected, wherein the respective counts for presentations and selections of search results that refer to documents in the base corpus are for searches initiated by users in a plurality of different countries who employ a specific language;

determining, for the plurality of search results responsive to the query, a respective count of times search results in the plurality of search results that refer to documents in a second corpus have been presented, and a respective count of times search results in the plurality of search results that refer to documents in the second corpus have been selected, wherein the respective counts for presentations and selections of search results that refer to documents in the second corpus are for searches initiated by users in the plurality of different countries who employ the specific language;

calculating a click through rate of the base corpus for the query based at least in part on the respective counts for the base corpus;

calculating a click through rate of the second corpus for the query based at least in part on the respective counts for the second corpus;

calculating a measure of relative relevance based at least in part on a ratio of the second corpus click through rate to the base corpus click through rate; and

providing the measure of relative relevance to a ranking engine for ranking of search results for a search corresponding to the query; and

wherein fewer search results of the plurality refer to documents in the second corpus than to documents in the base corpus.

10. The system of claim 9 wherein the respective counts for the base and second corpora are derived from language-specific data or country-specific data, and wherein calculating a click through rate for one of the corpora further comprises selecting which data to use for the respective counts based on a determination that the selected data will provide a reliable statistic.

11. The system of claim 9 wherein the base corpus comprises web pages and wherein the second corpus comprises videos.

12. The system of claim 9 wherein the base corpus comprises web pages and wherein the second corpus comprises images.

13. The system of claim 9 wherein the click through rate of the base corpus or of the second corpus is a ratio of the count of selections to the count of presentations.

14. The system of claim 9 wherein the ranking engine is configured to increase a score of a search result for the query that refers to a document in the second corpus based at least in part on the measure of relative relevance.

15. The system of claim 9 wherein the respective counts for the base and second corpora are derived from language-specific data and country-specific data, and wherein calculating a click through rate for one of the corpora is based on a weighted combination of the language-specific data and the country-specific data.

16. The system of claim 15 wherein an amount of the language-specific data is greater than an amount of the country-specific data.

17. A storage device encoded with a program product, the program product which, when executed by data processing apparatus, cause the data processing apparatus to perform operations comprising:

determining, for a plurality of search results responsive to a query, a respective count of times search results in the plurality of search results that refer to documents in a base corpus have been presented, and a respective count of times search results in the plurality of search results that refer to documents in the base corpus have been selected, wherein the respective counts for presentations and selections of search results that refer to documents in the base corpus are for searches initiated by users in a plurality of different countries who employ a specific language;

determining, for the plurality of search results responsive to the query, a respective count of times search results in the plurality of search results that refer to documents in a second corpus have been presented, and a respective count of times search results in the plurality of search results that refer to documents in the second corpus have been selected, wherein the respective counts for presentations and selections of search results that refer to documents in the second corpus are for searches initiated by users in the plurality of different countries who employ the specific language;

calculating a click through rate of the base corpus for the query based at least in part on the respective counts for the base corpus;

calculating a click through rate of the second corpus for the query based at least in part on the respective counts for the second corpus;

calculating a measure of relative relevance based at least in part on a ratio of the second corpus click through rate to the base corpus click through rate; and

providing the measure of relative relevance to a ranking engine for ranking of search results for a search corresponding to the query; and

wherein fewer search results of the plurality refer to documents in the second corpus than to documents in the base corpus.

18. The storage device of claim 17 wherein the respective counts for the base and second corpora are derived from language-specific data or country-specific data, and wherein calculating a click through rate for one of the corpora further comprises selecting which data to use for the respective counts based on a determination that the selected data will provide a reliable statistic.

19. The storage device of claim 17 wherein the base corpus comprises web pages and wherein the second corpus comprises videos.

20. The storage device of claim 17 wherein the base corpus comprises web pages and wherein the second corpus comprises images.

21. The storage device of claim 17 wherein the click through rate of the base corpus or of the second corpus is a ratio of the count of selections to the count of presentations.

22. The storage device of claim 17 wherein the ranking engine is configured to increase a score of a search result for the query that refers to a document in the second corpus based at least in part on the measure of relative relevance.

23. The storage device of claim 17 wherein the respective counts for the base and second corpora are derived from language-specific data and country-specific data, and wherein calculating a click through rate for one of the corpora is based on a weighted combination of the language-specific data and the country-specific data.

24. The storage device of claim 23 wherein an amount of the language-specific data is greater than an amount of the country-specific data.