Rank-based estimate of relevance values

• US 20040267717A1
• Filed: 06/27/2003
• Published: 12/30/2004
• Est. Priority Date: 06/27/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: SBC, Inc. (Illinois) (Wintrust Financial Corporation)
• Inventors: Slackman, Richard O.
• Granted Patent: US 7,206,780 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A ranked list of search results is received from a search engine based on a search query. A relevance value of a particular search result in the ranked list is estimated based on its rank and actual relevance values and ranks of at least two others of the search results.

Claims

1. A method comprising:

receiving a ranked list of search results from a search engine based on a search query; and

estimating a relevance value of a particular search result in the ranked list based on its rank and actual relevance values and ranks of at least two others of the search results.

2. The method of claim 1 wherein said estimating comprises fitting a curve, to represent relevance as a function of rank, to the actual relevance values and the ranks of the at least two others of the search results.

3. The method of claim 2 wherein said estimating further comprises evaluating the curve at the rank of the particular search result to estimate the relevance value.

4. The method of claim 3 wherein the curve is a line.

5. The method of claim 1 wherein said estimating comprises determining an interpolation function, to represent relevance as a function of rank, for the actual relevance values and the ranks of the at least two others of the search results.

6. The method of claim 5 wherein said estimating further comprises evaluating the interpolation function at the rank of the particular search result to estimate the relevance value.

7. The method of claim 1 wherein said estimating comprises linearly interpolating between two actual relevance values whose ranks bracket the rank of the particular search result.

8. The method of claim 1 wherein the actual relevance values are supplied by the search engine.

9. The method of claim 1 wherein the actual relevance values are not supplied by the search engine.

10. The method of claim 9 further comprising:

determining a first actual relevance value for a most-relevant one of the search results; and

determining a second actual relevance value for a least-relevant one of the search results;

wherein said estimating comprises linearly interpolating between the first actual relevance value and the second actual relevance value.

11. A method of weighting search results from a search engine based on a search query, the method comprising:

determining a plurality of categories associated with the search query;

for each of the categories, determining an associated category weighting value for the search engine;

determining a first associated relevance value for each of the categories based on the search query and one or more query terms associated with the category; and

determining a weighting value based on the category weighting values and the first associated relevance values.

12. The method of claim 11 further comprising:

determining a second associated relevance value for each of the categories by dividing its first associated relevance value by a sum of all of the first associated relevance values.

13. The method of claim 12 wherein said determining the weighting value comprises determining the weighting value based on a sum, over the categories, of each product of the associated category weighting value and the second associated relevance value.

14. A method comprising:

submitting a search query to a plurality of search engines;

receiving, from each of the plurality of search engines, an associated ranked list of search results based on the search query;

receiving a plurality of actual relevance values for a plurality of the search results based on the search query;

for at least one of the search results absent the actual relevance value, estimating its relevance value based on its rank, and the ranks and the actual relevance values of at least two others of the search results;

determining, for each of the plurality of search engines, an associated weighting value;

determining, for each of the ranked lists, an associated weighted relevance value for each of its search results based on the estimated relevance value or the actual relevance value of the search result and the weighting value associated with the search engine that provided the ranked list;

combining the ranked lists into a single list; and

sorting the search results in the single list based on the associated weighted relevance values.

15. The method of claim 14 wherein the actual values comprise normalized, search-engine-supplied relevance values.

16. The method of claim 14 wherein said determining the associated weighting value for a search engine comprises:

determining a plurality of categories associated with the search query;

determining an associated category search engine weighting value for each of the categories;

determining a first associated relevance value for each of the categories based on the search query and one or more query terms associated with the category;

determining a second associated relevance value for each of the categories by dividing its first associated relevance value by a sum of all first associated relevance values; and

determining the associated weighting value based on a sum, over the categories, of each product of the associated category search engine weighting value and the second associated relevance value.

17. The method of claim 14 wherein said estimating comprises:

fitting a curve, to represent relevance as a function of rank, to the actual relevance values and the ranks of the at least two others of the search results; and

evaluating the curve at the rank of the particular search result to estimate the relevance value.

18. The method of claim 14 wherein said estimating comprises:

determining an interpolation function, to represent relevance as a function of rank, for the actual relevance values and the ranks of the at least two others of the search results; and

evaluating the interpolation function at the rank of the particular search result to estimate the relevance value.

19. The method of claim 14 wherein the actual relevance values are search-engine-supplied.

20. The method of claim 14 wherein the actual relevance values are not search-engine-supplied.

21. An apparatus comprising:

a computer programmed to perform acts of;

receiving a ranked list of search results from a search engine based on a search query; and

estimating a relevance value of a particular search result in the ranked list based on its rank and actual relevance values and ranks of at least two others of the search results.

22. The apparatus of claim 21 wherein said estimating comprises fitting a curve, to represent relevance as a function of rank, to the actual relevance values and the ranks of the at least two others of the search results.

23. The apparatus of claim 22 wherein said estimating further comprises evaluating the curve at the rank of the particular search result to estimate the relevance value.

24. The apparatus of claim 23 wherein the curve is a line.

25. The apparatus of claim 21 wherein said estimating comprises determining an interpolation function, to represent relevance as a function of rank, for the actual relevance values and the ranks of the at least two others of the search results.

26. The apparatus of claim 25 wherein said estimating further comprises evaluating the interpolation function at the rank of the particular search result to estimate the relevance value.

27. The apparatus of claim 21 wherein said estimating comprises linearly interpolating between two actual relevance values whose ranks bracket the rank of the particular search result.

28. The apparatus of claim 21 wherein the actual relevance values are supplied by the search engine.

29. The apparatus of claim 21 wherein the actual relevance values are not supplied by the search engine.

30. The apparatus of claim 29 wherein the computer is programmed to perform further acts of:

determining a first actual relevance value for a most-relevant one of the search results; and

determining a second actual relevance value for a least-relevant one of the search results;

wherein said estimating comprises linearly interpolating between the first actual relevance value and the second actual relevance value.

31. An apparatus for weighting search results from a search engine based on a search query, the apparatus comprising:

a computer programmed to perform acts of;

determining a plurality of categories associated with the search query;

for each of the categories, determining an associated category weighting value for the search engine;

determining a first associated relevance value for each of the categories based on the search query and one or more query terms associated with the category; and

determining a weighting value based on the category weighting values and the first associated relevance values.

32. The apparatus of claim 31 wherein the computer is programmed to perform a further act of:

determining a second associated relevance value for each of the categories by dividing its first associated relevance value by a sum of the first associated relevance values.

33. The apparatus of claim 32 wherein said determining the weighting value comprises determining the weighting value based on a sum, over the categories, of each product of the associated category weighting value and the second associated relevance value.

34. An apparatus comprising:

a computer programmed to perform acts of;

submitting a search query to a plurality of search engines;

receiving, from each of the plurality of search engines, an associated ranked list of search results based on the search query;

receiving a plurality of actual relevance values for a plurality of the search results based on the search query;

for at least one of the search results absent an actual relevance value, estimating its relevance value based on its rank, and the ranks and the actual relevance values of at least two others of the search results;

determining, for each of the plurality of search engines, an associated weighting value;

determining, for each of the ranked lists, an associated weighted relevance value for each of its search results based on the estimated relevance value or the actual relevance value of the search result and the weighting value associated with the search engine that provided the ranked list;

combining the ranked lists into a single list; and

sorting the search results in the single list based on the associated weighted relevance values.

35. The apparatus of claim 34 wherein the actual values comprise normalized, search-engine-supplied relevance values.

36. The apparatus of claim 34 wherein said determining the associated weighting value for a search engine comprises:

determining a plurality of categories associated with the search query;

determining an associated category search engine weighting value for each of the categories;

determining a first associated relevance value for each of the categories based on the search query and one or more query terms associated with the category;

determining a second associated relevance value for each of the categories by dividing its first associated relevance value by a sum of all first associated relevance values; and

determining the associated weighting value based on a sum, over the categories, of each product of the associated category search engine weighting value and the second associated relevance value.

37. The apparatus of claim 34 wherein said estimating comprises:

fitting a curve, to represent relevance as a function of rank, to the actual relevance values and the ranks of the at least two others of the search results; and

evaluating the curve at the rank of the particular search result to estimate the relevance value.

38. The apparatus of claim 34 wherein said estimating comprises:

determining an interpolation function, to represent relevance as a function of rank, for the actual relevance values and the ranks of the at least two others of the search results; and

evaluating the interpolation function at the rank of the particular search result to estimate the relevance value.

39. The apparatus of claim 34 wherein the actual relevance values are search-engine-supplied.

40. The apparatus of claim 34 wherein the actual relevance values are not search-engine-supplied.

41. An article comprising:

a computer-readable medium having computer-readable program code to cause a computer to perform acts of;

receiving a ranked list of search results from a search engine based on a search query; and

estimating a relevance value of a particular search result in the ranked list based on its rank and actual relevance values and ranks of at least two others of the search results.

42. The article of claim 41 wherein said estimating comprises fitting a curve, to represent relevance as a function of rank, to the actual relevance values and the ranks of the at least two others of the search results.

43. The article of claim 42 wherein said estimating further comprises evaluating the curve at the rank of the particular search result to estimate the relevance value.

44. The article of claim 43 wherein the curve is a line.

45. The article of claim 41 wherein said estimating comprises determining an interpolation function, to represent relevance as a function of rank, for the actual relevance values and the ranks of the at least two others of the search results.

46. The article of claim 45 wherein said estimating further comprises evaluating the interpolation function at the rank of the particular search result to estimate the relevance value.

47. The article of claim 41 wherein said estimating comprises linearly interpolating between two actual relevance values whose ranks bracket the rank of the particular search result.

48. The article of claim 41 wherein the actual relevance values are supplied by the search engine.

49. The article of claim 41 wherein the actual relevance values are not supplied by the search engine.

50. The article of claim 49 wherein the computer-readable program code further causes the computer to perform acts of:

determining a first actual relevance value for a most-relevant one of the search results; and

determining a second actual relevance value for a least-relevant one of the search results;

wherein said estimating comprises linearly interpolating between the first actual relevance value and the second actual relevance value.

51. An article for weighting search results from a search engine based on a search query, the article comprising:

a computer-readable medium having computer-readable program code to cause a computer to perform acts of;

determining a plurality of categories associated with the search query;

for each of the categories, determining an associated category weighting value for the search engine;

determining a first associated relevance value for each of the categories based on the search query and one or more query terms associated with the category; and

determining a weighting value based on the category weighting values and the first associated relevance values.

52. The article of claim 51 wherein the computer-readable program code further causes the computer to perform an act of:

determining a second associated relevance value for each of the categories by dividing its first associated relevance value by a sum of all first associated relevance values.

53. The article of claim 52 wherein said determining the weighting value comprises determining the weighting value based on a sum, over the categories, of each product of the associated category weighting value and the second associated relevance value.

54. An article comprising:

a computer-readable medium having computer-readable program code to cause a computer to perform acts of;

submitting a search query to a plurality of search engines;

receiving, from each of the search engines, an associated ranked list of search results based on the search query;

receiving a plurality of actual relevance values for a plurality of the search results based on the search query;

for at least one of the plurality of search results absent an actual relevance value, estimating its relevance value based on its rank, and the ranks and the actual relevance values of at least two others of the search results;

determining, for each of the plurality of search engines, an associated weighting value;

determining, for each of the ranked lists, an associated weighted relevance value for each of its search results based on the estimated relevance value or the actual relevance value of the search result and the weighting value associated with the search engine that provided the ranked list;

combining the ranked lists into a single list; and

sorting the search results in the single list based on the associated weighted relevance values.

55. The article of claim 54 wherein the actual relevance values comprise normalized, search-engine-supplied relevance values.

56. The article of claim 54 wherein said determining the associated weighting value for a search engine comprises:

determining a plurality of categories associated with the search query;

determining an associated category search engine weighting value for each of the categories;

determining a first associated relevance value for each of the categories based on the search query and one or more query terms associated with the category;

determining a second associated relevance value for each of the categories by dividing its first associated relevance value by a sum of all first associated relevance values; and

determining the associated weighting value based on a sum, over the categories, of each product of the associated category search engine weighting value and the second associated relevance value.

57. The article of claim 54 wherein said estimating comprises:

fitting a curve, to represent relevance as a function of rank, to the actual relevance values and the ranks of the at least two others of the search results; and

evaluating the curve at the rank of the particular search result to estimate the relevance value.

58. The article of claim 54 wherein said estimating comprises:

determining an interpolation function, to represent relevance as a function of rank, for the actual relevance values and the ranks of the at least two others of the search results; and

evaluating the interpolation function at the rank of the particular search result to estimate the relevance value.

59. The article of claim 54 wherein the actual relevance values are search-engine-supplied.

60. The article of claim 54 wherein the actual relevance values are not search-engine-supplied.