Rank-based estimate of relevance values

• US 20070156663A1
• Filed: 02/28/2007
• Published: 07/05/2007
• Est. Priority Date: 06/27/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: SBC Knowledge Ventures LP (AT&T, Inc.)
• Inventors: Slackman, Richard
• Granted Patent: US 7,716,202 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A ranked list of search results is received from a search engine based on a search query. A relevance value of a particular search result in the ranked list is estimated based on its rank and based on relevance values and ranks of at least two others of the search results.

Claims

1. A method, comprising:

receiving a first ranked list of search results from a first search engine based on a search query; and

estimating a relevance value of a particular search result in the first ranked list based on its rank and based on relevance values and ranks of at least two others of the search results.

2. The method of claim 1, further comprising receiving the search query from an end-user computer via a network.

3. The method of claim 2, further comprising submitting the search query to the first search engine.

4. The method of claim 3, further comprising:

submitting the search query to a plurality of search engines that includes the first search engine;

receiving a plurality of ranked lists that includes the first ranked list, each of the plurality of ranked lists received from the plurality of search engines, wherein each of the plurality of ranked lists includes at least one search result;

producing a combined ranked list that includes the search results of the plurality of ranked lists; and

estimating a relevance value of a particular search result in the combined ranked list based on its rank and based on relevance values and ranks of at least two other search results in the combined ranked list.

5. The method of claim 4, further comprising normalizing relevance values of search results in each of the plurality of ranked lists before producing the combined ranked list.

6. The method of claim 1, wherein said estimating comprises fitting a curve, to represent relevance as a function of rank, to the actual relevance values and the ranks of the at least two others of the search results.

7. The method of claim 6, wherein said estimating further comprises evaluating the curve at the rank of the particular search result to estimate the relevance value.

8. The method of claim 7, wherein the curve is a line.

9. The method of claim 1, wherein said estimating comprises determining an interpolation function, to represent relevance as a function of rank, for the actual relevance values and the ranks of the at least two others of the search results.

10. The method of claim 9, wherein said estimating further comprises evaluating the interpolation function at the rank of the particular search result to estimate the relevance value.

11. The method of claim 1, wherein said estimating comprises linearly interpolating between two actual relevance values whose ranks bracket the rank of the particular search result.

12. The method of claim 1, wherein the actual relevance values are supplied by the search engine.

13. The method of claim 1, wherein the actual relevance values are not supplied by the search engine.

14. The method of claim 13, further comprising:

determining a first actual relevance value for a most-relevant one of the search results; and

determining a second actual relevance value for a least-relevant one of the search results;

wherein said estimating comprises linearly interpolating between the first actual relevance value and the second actual relevance value.

15. A method of weighting search results from a search engine, the method comprising:

determining a plurality of categories associated with a search query;

for each of the categories, determining an associated category weighting value associated with the search engine;

determining a first associated relevance value for each of the categories based on the search query and one or more query terms associated with the category; and

determining a weighting value based on the category weighting values and the first associated relevance values.

16. The method of claim 15, further comprising determining a second associated relevance value for each of the categories by dividing its first associated relevance value by a sum of all of the first associated relevance values.

17. The method of claim 16, wherein said determining the weighting value comprises determining the weighting value based on a sum, over the categories, of each product of the associated category weighting value and the second associated relevance value.

18. A method, comprising:

submitting a search query to a plurality of search engines;

receiving, from each of the plurality of search engines, an associated ranked list of search results based on the search query;

receiving a plurality of actual relevance values for a plurality of the search results based on the search query;

for at least one of the search results absent the actual relevance value, estimating its relevance value based on its rank, and the ranks and the actual relevance values of at least two others of the search results;

determining, for each of the plurality of search engines, an associated weighting value;

determining, for each of the ranked lists, an associated weighted relevance value for each of its search results based on the estimated relevance value or the actual relevance value of the search result and the weighting value associated with the search engine that provided the ranked list;

combining the ranked lists into a single list; and

sorting the search results in the single list based on the associated weighted relevance values.

19. The method of claim 18, wherein the actual values comprise normalized, search-engine-supplied relevance values.

20. The method of claim 18, wherein said determining the associated weighting value for a search engine comprises:

determining a plurality of categories associated with the search query;

determining an associated category search engine weighting value for each of the categories;

determining a first associated relevance value for each of the categories based on the search query and one or more query terms associated with the category;

determining a second associated relevance value for each of the categories by dividing its first associated relevance value by a sum of all first associated relevance values; and

determining the associated weighting value based on a sum, over the categories, of each product of the associated category search engine weighting value and the second associated relevance value.

21. The method of claim 18, wherein said estimating comprises:

fitting a curve, to represent relevance as a function of rank, to the actual relevance values and the ranks of the at least two others of the search results; and

evaluating the curve at the rank of the particular search result to estimate the relevance value.

22. The method of claim 18, wherein said estimating comprises:

determining an interpolation function, to represent relevance as a function of rank, for the actual relevance values and the ranks of the at least two others of the search results; and

evaluating the interpolation function at the rank of the particular search result to estimate the relevance value.

23. The method of claim 18, wherein the actual relevance values are search-engine-supplied.

24. The method of claim 18, wherein the actual relevance values are not search-engine-supplied.