Categorization of search results

• US 8,498,984 B1
• Filed: 11/21/2011
• Issued: 07/30/2013
• Est. Priority Date: 11/21/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lee, Junyoung, Hwang, Yong Soo, Kang, Jaeho, Lee, Jungyeol, Seo, Minkoo
• Primary Examiner(s): HO, BINH VAN
• Application Number:
  US13/301,228
• Time in Patent Office: 617 Days
• Field of Search: 707/731
• US Class Current: 707/731
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

A server is configured to receive a search query from a client device; identify documents as search results relevant to the search query, where each of the search results has a respective first score; rank the search results, based on the respective first scores, to form a ranked list of search results, where each of the search results has a respective position, within the ranked list, that is based on the ranking; determine a selection rate corresponding to each of the positions in the ranked list; identify categories with which the search results are associated; generate second scores for the categories based on the selection rates; select search results for the selected categories; generate a search result document that includes information regarding the selected categories and information regarding the selected search results; and provide the search result document for display on the client device.

Claims

1. A method performed by one or more server devices, the method comprising:

receiving, by at least one of the one or more server devices, a search query from a client device;

identifying, by at least one of the one or more server devices, documents as search results relevant to the search query, where each of the search results has a respective first score;

ranking, by at least one of the one or more server devices, the search results, based on the respective first scores, to form a ranked list of search results, where each of the search results has a respective position, within the ranked list, that is based on the ranking;

determining, by at least one of the one or more server devices, a selection rate corresponding to each of the positions in the ranked list,

a particular selection rate, corresponding to a particular position, being determined based on selections from the particular position, and

determining the selection rate for one of the positions in the ranked list including;

selectively;

obtaining information regarding an actual selection rate, corresponding to the one of the positions, from observed selections, or

estimating a selection rate corresponding to the one of the positions, and

using the actual selection rate or the estimated selection rate as the selection rate for the one of the positions;

identifying, by at least one of the one or more server devices, categories with which the search results are associated;

generating, by at least one of the one or more server devices, second scores for the categories based on the selection rates, where the second score, for a particular one of the categories, is based on the selection rate corresponding to the position, in the ranked list, of one or more of the search results that are associated with the particular one of the categories, generating the second scores for the categories including;

generating the second score, for one of the categories,

based on a combination of a plurality of selection rates corresponding to positions, in the ranked list, of a set of the search results that are associated with the one of the categories;

selecting, by at least one of the one or more server devices, a plurality of the categories based on the second scores of the categories;

selecting, by at least one of the one or more server devices, a plurality of the search results for the selected plurality of categories;

generating, by at least one of the one or more server devices, a search result document that includes information regarding the selected plurality of categories and information regarding the selected plurality of search results; and

providing, by at least one of the one or more server devices, the search result document for display on the client device.

2. The method of claim 1, where the first score, for one of the search results, is based on at least one of:

a measure of relevance of the one of the search results to the search query, or

a measure of quality of the one of the search results.

3. The method of claim 1, where identifying the categories includes:

performing a look-up in a memory device to identify the category with which one of the search results is associated, where the memory device stores information associating documents with categories.

4. The method of claim 1, where identifying the categories includes:

classifying one of the documents to assign the search result, corresponding to the one of the documents, to one of the categories.

5. The method of claim 1, further comprising:

comparing the respective first scores to a threshold value; and

removing one of the search results, that has a respective first score that is less than the threshold value, before generating the second scores for the categories, where the second scores are generated without using information regarding the removed one of the search results.

6. The method of claim 1, where selecting the plurality of search results includes:

selecting all of the search results associated with a set of the positions in the ranked list, where the set of the positions corresponds to a plurality of top positions in the ranked list.

7. The method of claim 1, where generating the search result document includes:

organizing the selected plurality of categories within the search result document, where a first one of the selected plurality of categories is presented at a particular location within the search result document irrespective of the second score of the first one of the selected plurality of categories, and where a second one of the selected plurality of categories and a third one of the selected plurality of categories are presented, below the first one of the selected plurality of categories, based on the second scores of the second one of the selected plurality of categories and the third one of the selected plurality of categories.

8. A system, comprising:

one or more server devices to;

receive a search query from a client device;

identify documents as search results relevant to the search query, where each of the search results has a respective first score;

rank the search results, based on the respective first scores, to form a ranked list of search results, where each of the search results has a respective position, within the ranked list, that is based on the ranking;

determine a selection rate corresponding to each of the positions in the ranked list,

a particular selection rate, corresponding to a particular position, being determined based on selections from the particular position, and

the one or more devices, when determining the selection rate for one of the positions in the ranked list, being further to;

selectively;

 

obtain information regarding an actual selection rate, corresponding to the one of the positions, from observed selections, or

 

estimate a selection rate corresponding to the one of the positions, and

use the actual selection rate or the estimated selection rate as the selection rate for the one of the positions;

identify categories with which the search results are associated;

generate second scores for the categories based on the selection rates, where the second score, for a particular one of the categories, is based on the selection rate corresponding to the position, in the ranked list, of one or more of the search results that are associated with the particular one of the categories,

the one or more server devices generating the second score, for one of the categories, based on a combination of a plurality of selection rates corresponding to positions, in the ranked list, of a set of the search results that are associated with the one of the categories;

select a plurality of the categories based on the second scores of the categories;

select a plurality of the search results for the selected plurality of categories;

generate a search result document that includes information regarding the selected plurality of categories and information regarding the selected plurality of search results; and

provide the search result document for display on the client device.

9. The system of claim 8, where, when selecting the plurality of categories, at least one of the one or more server devices is to:

select a first one of the categories irrespective of the second score of the first one of the categories, and

select a second one of the categories based on the second score of the second one of the categories, and

where, when generating the search result document, at least one of the one or more server devices is to include information regarding the first one of the categories and the second one of the categories in the search result document.

10. The system of claim 8, where, when selecting the plurality of search results, at least one of the one or more server devices is to:

select a particular quantity of the search results, associated with one of the categories, when a quantity of the search results, associated with the one of the categories, is greater than the particular quantity, and

select all of the search results, associated with the one of the categories, when the quantity of the search results, associated with the one of the categories, is less than or equal to the particular quantity.

11. The system of claim 8, where, when generating the search result document, at least one of the one or more server devices is to:

organize the selected plurality of search results based on the selected plurality of categories, where a first one of the selected plurality of search results is presented adjacent to a second one of the selected plurality of search results, where the first one of the selected plurality of search results and the second one of the selected plurality of search results are both associated with a same one of the selected plurality of categories.

12. The system of claim 11, where at least one of the one or more server devices is further to:

provide an option to remove the organization of the selected plurality of search results based on the selected plurality of categories;

receive selection of the option;

generate, in response to the selection of the option, another search result document that includes the ranked list of the search results, where the first one of the selected plurality of search results is not presented adjacent to the second one of the selected plurality of search results in the other search result document; and

provide the other search result document for display on the client.

13. The system of claim 8, where at least one of the one or more server devices is further to:

compare the respective first scores to a threshold value; and

remove one of the search results, that has a respective first score that is less than the threshold value, before generating the second scores for the categories, where the second scores are generated without using information regarding the removed one of the search results.

14. A non-transitory computer-readable medium, comprising:

a plurality of instructions which, when executed by one or more processors, cause the one or more processors to;

receive a search query from a client device;

identify documents as search results relevant to the search query, where each of the search results has a respective first score;

rank the search results, based on the respective first scores, to form a ranked list of search results, where each of the search results has a respective position, within the ranked list, that is based on the ranking;

determine a selection rate corresponding to each of the positions in the ranked list,

a particular selection rate, corresponding to a particular position, being determined based on selections from the particular position, and

the selection rate for one of the positions in the ranked list, being determined based on;

selectively;

 

obtaining information regarding an actual selection rate, corresponding to the one of the positions, from observed selections, or

 

estimating a selection rate corresponding to the one of the positions, and

using the actual selection rate or the estimated selection rate as the selection rate for the one of the positions;

identify categories with which the search results are associated;

generate second scores for the categories based on the selection rates, where the second score, for a particular one of the categories, is based on the selection rate corresponding to the position, in the ranked list, of one or more of the search results that are associated with the particular one of the categories,

the second score, for one of the categories, being generated based on a combination of a plurality of selection rates corresponding to positions, in the ranked list, of a set of the search results that are associated with the one of the categories;

select a plurality of the categories based on the second scores of the categories;

select a plurality of the search results for the selected plurality of categories;

generate a search result document that includes information regarding the selected plurality of categories and information regarding the selected plurality of search results; and

provide the search result document for display on the client device.

15. The non-transitory computer-readable medium of claim 14, where the plurality of instructions which, when executed by the one or more processors, further cause the one or more processors to:

compare the respective first scores to a threshold value; and

remove one of the search results, that has a respective first score that is less than the threshold value, before generating the second scores for the categories, where the second scores are generated without using information regarding the removed one of the search results.

16. The non-transitory computer-readable medium of claim 14, where one or more instructions, of the plurality of instructions, to select the plurality of categories cause the one or more processors to:

select a first one of the categories irrespective of the second score of the first one of the categories, and

select a second one of the categories based on the second score of the second one of the categories, and

where one or more instructions, of the plurality of instructions, to generate the search result document cause the one or more processors to;

include information regarding the first one of the categories and the second one of the categories in the search result document.

17. The non-transitory computer-readable medium of claim 14, where one or more instructions, of the plurality of instructions, to select the plurality of search results cause the one or more processors to:

select a particular quantity of the search results, associated with one of the categories, when a quantity of the search results, associated with the one of the categories, is greater than the particular quantity, and

select all of the search results, associated with the one of the categories, when the quantity of the search results, associated with the one of the categories, is less than or equal to the particular quantity.

18. The non-transitory computer-readable medium of claim 14, where one or more instructions, of the plurality of instructions, to select the plurality of search results cause the one or more processors to:

select all of the search results associated with a set of the positions in the ranked list, where the set of the positions corresponds to a plurality of top positions in the ranked list.

19. The non-transitory computer-readable medium of claim 14, where one or more instructions, of the plurality of instructions, to generate the search result document cause the one or more processors to:

organize the selected plurality of search results based on the selected plurality of categories, where a first one of the selected plurality of search results is presented adjacent to a second one of the selected plurality of search results, where the first one of the selected plurality of search results and the second one of the selected plurality of search results are both associated with a same one of the selected plurality of categories.

20. The non-transitory computer-readable medium of claim 19, where the plurality of instructions which, when executed by the one or more processors, further cause the one or more processors to:

provide an option to remove the organization of the selected plurality of search results based on the selected plurality of categories;

receive selection of the option;

generate, in response to the selection of the option, another search result document that includes the ranked list of the search results, where the first one of the selected plurality of search results is not presented adjacent to the second one of the selected plurality of search results within the other search result document; and

provide the other search result document for display on the client.

21. The non-transitory computer-readable medium of claim 14, where one or more instructions, of the plurality of instructions, to generate the search result document cause the one or more processors to:

organize the selected plurality of categories within the search result document, where a first one of the selected plurality of categories is presented at a particular location within the search result document irrespective of the second score of the first one of the selected plurality of categories, and where a second one of the selected plurality of categories and a third one of the selected plurality of categories are presented, below the first one of the selected plurality of categories, based on the second scores of the second one of the selected plurality of categories and the third one of the selected plurality of categories.