Temporal search results

• US 20060248073A1
• Filed: 04/28/2005
• Published: 11/02/2006
• Est. Priority Date: 04/28/2005
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Jones, Rosie, Diaz, Fernando
• Granted Patent: US 7,577,651 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3328 using graphical result space presentation or visualisation
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

In certain implementations, a system and/or method is provided for providing search results in response to a search query. In this implementation a temporal profile of the search query is built from temporal data associated with documents retrieved in response to the search query. From features of the temporal profile it may determined whether the search query would benefit from relevance feedback from a user. If there is a determination that the search query will benefit from the relevance feedback, relevance feedback is sought from the user. Search results are provided based on the relevance feedback.

Claims

1. A method for providing search results in response to a search query, the method comprising:

building a temporal profile of the search query from temporal data associated with documents retrieved in response to the search query;

determining from features of the temporal profile whether the search query would benefit from relevance feedback from a user;

seeking the relevance feedback from the user if there is a determination that the search query will benefit from the relevance feedback from the user; and

providing search results based on the relevance feedback.

2. The method of claim 1 wherein determining comprises classifying the search query based on the temporal profile of the search query.

3. The method of claim 2 wherein determining comprises using a combination of features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

4. The method of claim 2 wherein determining comprises using a combination of features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

5. The method of claim 1 wherein determining comprises predicting an average precision of the search query using a combination of the features of the temporal profile.

6. The method of claim 5 wherein determining comprises predicting an average precision of the search query using a combination of the features of the temporal profile and a content feature.

7. The method of claim 6 wherein determining comprises using a combination of features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

8. The method of claim 6 wherein determining comprises using a combination of features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

9. The method of claim 8 wherein determining comprises using a content clarity.

10. The method of claim 8 wherein determining comprises using the Kullback-Leibler divergence.

11. The method of claim 10 wherein determining comprises using a content clarity.

12. The method of claim 6 wherein predicting the average precision comprises using a predictive model.

13. The method of claim 12 wherein predicting the average precision comprises using a learning algorithm.

14. The method of claim 13 wherein predicting the average precision comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

15. The method of claim 1 wherein seeking relevance feedback from the user further comprises constructing a display capable of being displayed at an interactive user interface, the display incorporating an indication of a temporal nature of the search query.

16. The method of claim 15 wherein constructing a display comprises a constructing a timeline comprising temporal event indicators and corresponding event summaries.

17. The method of claim 16 wherein constructing a display comprises constructing a non-extractive event summary.

18. The method of claim 16 wherein constructing a display comprises constructing a sentence likelihood event summary.

19. The method of claim 16 wherein constructing a display comprises constructing an extractive event summary.

20. The method of claim 15 wherein constructing a display comprises a constructing a listing.

21. The method of claim 1 wherein building temporal profile comprises smoothing the temporal data.

22. The method of claim 1 wherein providing search results comprises modifying an algorithm for retrieving documents.

23. A method for providing search results in response to a search query, the method comprising:

building a temporal profile of the search query from temporal data associated with documents retrieved in response to the search query;

classifying the search query using the temporal features of the temporal profile; and

constructing an interactive display capable of being presented to a user for obtaining query specific relevance feedback from the user if the search query is classified as temporally ambiguous.

24. The method of claim 23 wherein classifying the temporal profile comprises using features of the temporal profile.

25. The method of claim 24 wherein classifying comprises using a combination of features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

26. The method of claim 24 wherein classifying comprises using a combination of features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

27. The method of claim 24 wherein classifying comprises using a Kullback-Leibler divergence.

28. The method of claim 23 wherein constructing the interactive display comprises constructing a timeline comprising:

a) temporal event indicators; and

b) temporal event summaries corresponding with the temporal event indicators.

29. The method of claim 23 wherein predicting the average precision comprises using a learning algorithm.

30. The method of claim 29 wherein predicting the average precision comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

31. A method for providing search results in response to a search query, the method comprising:

building a temporal profile of the search query from temporal data associated with documents retrieved in response to the search query;

predicting an average precision of the search query using a combination of at least one temporal feature of the temporal profile and a content feature of the documents retrieved in response to the search query;

determining from the average precision whether the search query would benefit from relevance feedback from a user; and

constructing an interactive display capable of being presented to a user for obtaining the relevance feedback from the user if there is a determination that the search query will benefit from the relevance feedback from the user.

32. The method of claim 31 wherein constructing the interactive display comprises constructing a timeline comprising:

a) temporal event indicators; and

b) temporal event summaries corresponding with the temporal event indicators.

33. The method of claim 32 wherein predicting the average precision comprises using a combination of temporal features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

34. The method of claim 33 wherein predicting the average precision comprises using a combination of temporal features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

35. The method of claim 32 wherein predicting comprises using a Kullback-Leibler divergence.

36. The method of claim 31 wherein predicting the average precision comprises using a learning algorithm.

37. The method of claim 36 wherein predicting the average precision comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

38. A method for providing search results in response to a search query comprising:

receiving a search query;

building a temporal profile of the search query from temporal data associated with documents retrieved by a search engine in response to the search query;

performing at least one of;

(1) classifying the search query using the temporal features of the temporal profile;

or (2) predicting an average precision of the search query using a combination of at least one temporal feature of the temporal profile and a content feature of the documents retrieved in response to the search query;

determining from performing the at least one of classifying or predicting the average precision whether the search query would benefit from relevance feedback from a user;

constructing an interactive timeline for display to the user if there is a determination that the search query will benefit from the relevance feedback from the user;

modifying the algorithm for retrieving documents based on a selection by the user from the timeline; and

providing search results for display to the user using the modified algorithm for retrieving documents.

39. A method for increasing the relevance of search results comprising:

receiving a search query;

identifying a temporal profile of the search query from search results retrieved by a search engine in response to the search query;

determining a relevance of the temporal profile of the search query; and

influencing a presentation of the search results based on the temporal profile of the query.

40. The method of claim 39 wherein influencing comprises generating an interactive timeline based on the temporal profile of the search query.

41. The method of claim 40 wherein generating the interactive timeline comprises generating a timeline comprising:

a) temporal event indicators; and

b) temporal event summaries corresponding with the temporal event indicators.

42. The method of claim 40 further comprising receiving the temporal relevance feedback from a user input to the interactive timeline.

43. The method of claim 42 further comprising generating search results for presentation to the user based on the temporal relevance feedback received from the user.

44. The method of claim 43 wherein determining comprises predicting an average precision of the search query using a combination of at least one temporal feature of the temporal profile and a content feature of documents retrieved in response to the search query.

45. The method of claim 44 wherein predicting the average precision comprises using a combination of temporal features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

46. The method of claim 45 wherein predicting the average precision comprises using a combination of temporal features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

47. The method of claim 44 wherein predicting the average precision comprises using a Kullback-Leibler divergence.

48. The method of claim 44 wherein predicting the average precision comprises using a learning algorithm.

49. The method of claim 48 wherein predicting the average precision comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

50. The method of claim 39 wherein determining the relevance comprises classifying the temporal profile.

51. The method of claim 50 wherein classifying the temporal profile comprises determining whether the temporal profile indicates the search query is temporally ambiguous.

52. The method of claim 50 wherein classifying the temporal profile comprises determining whether the temporal profile indicates the search query is at least one of:

(a) temporally ambiguous;

(b) temporally unambiguous;

or atemporal.

53. The method of claim 50 wherein classifying the temporal profile comprises determining whether the temporal profile indicates the search query is not temporally unambiguous;

and not atemporal.

54. The method of claim 39 wherein influencing comprises influencing a rank order of the search results in a listing based on the temporal profile.

55. The method of claim 54 comprising receiving the temporal relevance feedback from a user input to the interactive timeline.

56. The method of claim 55 further comprising generating search results for presentation to the user based on the temporal relevance feedback received from the user.

57. The method of claim 39 wherein identifying the temporal profile further comprises smoothing the temporal data associated with a results set of documents.

58. The method of claim 39 wherein influencing comprises modifying an algorithm for retrieving documents.

59. A method for predicting precision of search queries comprising:

receiving a search query;

identifying a temporal profile of the search query from temporal data associated with documents retrieved in response to the search query;

determining temporal features of the temporal profile using at least one of;

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

predicting an average precision of the search query using a combination of at least one of the temporal features of the temporal profile and a content feature of the documents retrieved in response to the search query.

60. The method of claim 59 wherein predicting comprises using a learning algorithm.

61. The method of claim 60 wherein predicting comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

62. The method of claim 59 wherein identifying comprises using a Kullback-Leibler divergence.

63. The method of claim 59 wherein identifying the temporal profile further comprises smoothing the temporal data associated with a results set of documents.

64. A system for providing search results in response to a search query, the system comprising:

a) a computer adapted to receive a search query; and

b) the computer comprising a processor, the processor comprising processor executable instructions so as to be capable of;

building a temporal profile of the search query from temporal data associated with documents retrieved in response to the search query;

determining from features of the temporal profile whether the search query would benefit from relevance feedback from a user;

seeking relevance feedback from the user if there is a determination that the search query will benefit from the relevance feedback from the user; and

providing search results based on the relevance feedback.

65. The system of claim 64 wherein determining comprises classifying the search query based on the temporal profile of the search query.

66. The system of claim 65 wherein determining comprises using a combination of features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

67. The system of claim 65 wherein determining comprises using a combination of features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

68. The system of claim 64 wherein determining comprises predicting an average precision of the search query using a combination of the features of the temporal profile.

69. The system of claim 68 wherein determining comprises predicting an average precision of the search query using a combination of the features of the temporal profile and a content feature.

70. The system of claim 69 wherein determining comprises using a combination of features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

71. The system of claim 69 wherein determining comprises using a combination of features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

72. The system of claim 71 wherein determining comprises using a content clarity.

73. The system of claim 71 wherein determining comprises using the Kullback-Leibler divergence.

74. The system of claim 73 wherein determining comprises using a content clarity.

75. The system of claim 69 wherein predicting the average precision comprises using a predictive model.

76. The system of claim 75 wherein predicting the average precision comprises using a learning algorithm.

77. The system of claim 76 wherein predicting the average precision comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

78. The system of claim 64 wherein seeking relevance feedback from the user further comprises constructing a display capable of being displayed at an interactive user interface, the display incorporating an indication of a temporal nature of the search query.

79. The system of claim 78 wherein constructing a display comprises a constructing a timeline comprising temporal event indicators and corresponding event summaries.

80. The system of claim 79 wherein constructing a display comprises constructing a non-extractive event summary.

81. The system of claim 79 wherein constructing a display comprises constructing a sentence likelihood event summary.

82. The system of claim 79 wherein constructing a display comprises constructing an extractive event summary.

83. The system of claim 78 wherein constructing a display comprises constructing a listing.

84. The system of claim 64 wherein building temporal profile comprises smoothing the temporal data.

85. The system of claim 64 wherein providing search results comprises modifying an algorithm for retrieving documents.

86. A system for providing search results in response to a search query, the system comprising:

a) a computer adapted to receive a search query; and

b) the computer comprising a processor, the processor comprising processor executable instructions so as to be capable of;

building a temporal profile of the search query from temporal data associated with documents retrieved in response to the search query;

classifying the search query using the temporal features of the temporal profile; and

constructing an interactive display capable of being presented to a user for obtaining query specific relevance feedback from the user if the search query is classified as temporally ambiguous.

87. The system of claim 86 wherein classifying the temporal profile comprises using features of the temporal profile.

88. The system of claim 87 wherein classifying comprises using a combination of features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

89. The system of claim 87 wherein classifying comprises using a combination of features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

90. The system of claim 87 wherein classifying comprises using a Kullback-Leibler divergence.

91. The system of claim 86 wherein constructing the interactive display comprises constructing a timeline comprising:

a) temporal event indicators; and

b) temporal event summaries corresponding with the temporal event indicators.

92. The system of claim 86 wherein predicting the average precision comprises using a learning algorithm.

93. The system of claim 92 wherein predicting the average precision comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

94. A system for providing search results in response to a search query, the system comprising:

a) a computer adapted to receive a search query; and

b) the computer comprising a processor, the processor comprising processor executable instructions so as to be capable of;

building a temporal profile of the search query from temporal data associated with documents retrieved in response to the search query;

predicting an average precision of the search query using a combination of at least one temporal feature of the temporal profile and a content feature of the documents retrieved in response to the search query;

determining from the average precision whether the search query would benefit from relevance feedback from a user; and

constructing an interactive display capable of being presented to a user for obtaining the relevance feedback from the user if there is a determination that the search query will benefit from the relevance feedback from the user.

95. The system of claim 94 wherein constructing the interactive display comprises constructing a timeline comprising:

a) temporal event indicators; and

b) temporal event summaries corresponding with the temporal event indicators.

96. The system of claim 95 wherein predicting the average precision comprises using a combination of temporal features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

97. The system of claim 96 wherein predicting the average precision comprises using a combination of temporal features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

98. The system of claim 95 wherein predicting comprises using a Kullback-Leibler divergence.

99. The system of claim 94 wherein predicting the average precision comprises using a learning algorithm.

100. The system of claim 99 wherein predicting the average precision comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

101. A system for providing search results in response to a search query, the system comprising:

a) a computer adapted to receive a search query; and

b) the computer comprising a processor, the processor comprising processor executable instructions so as to be capable of;

receiving a search query;

building a temporal profile of the search query from temporal data associated with documents retrieved by a search engine in response to the search query;

performing at least one of;

(1) classifying the search query using the temporal features of the temporal profile;

or (2) predicting an average precision of the search query using a combination of at least one temporal feature of the temporal profile and a content feature of the documents retrieved in response to the search query;

determining from performing the at least one of classifying or predicting the average precision whether the search query would benefit from relevance feedback from a user;

constructing an interactive timeline for display to the user if there is a determination that the search query will benefit from the relevance feedback from the user;

modifying the algorithm for retrieving documents based on a selection by the user from the timeline; and

providing search results for display to the user using the modified algorithm for retrieving documents.

102. A system for providing search results in response to a search query, the system comprising:

a) a computer adapted to receive a search query; and

b) the computer comprising a processor, the processor comprising processor executable instructions so as to be capable of;

receiving a search query;

identifying a temporal profile of the search query from search results retrieved by a search engine in response to the search query;

determining a relevance of the temporal profile of the search query; and

influencing a presentation of the search results based on the temporal profile of the query.

103. The system of claim 102 wherein influencing comprises generating an interactive timeline based on the temporal profile of the search query.

104. The system of claim 103 wherein generating the interactive timeline comprises generating a timeline comprising:

a) temporal event indicators; and

b) temporal event summaries corresponding with the temporal event indicators.

105. The system of claim 103 further comprising receiving the temporal relevance feedback from a user input to the interactive timeline.

106. The system of claim 105 further comprising generating search results for presentation to the user based on the temporal relevance feedback received from the user.

107. The system of claim 106 wherein determining comprises predicting an average precision of the search query using a combination of at least one temporal feature of the temporal profile and a content feature of documents retrieved in response to the search query.

108. The system of claim 107 wherein predicting the average precision comprises using a combination of temporal features comprising:

(a) divergence;

(b) autocorrelation;

(c) statistics of the rank order;

or (d) burst model.

109. The system of claim 108 wherein predicting the average precision comprises using a combination of temporal features comprising:

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

110. The system of claim 107 wherein predicting the average precision comprises using a Kullback-Leibler divergence.

111. The system of claim 107 wherein predicting the average precision comprises using a learning algorithm.

112. The system of claim 111 wherein predicting the average precision comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

113. The system of claim 102 wherein determining the relevance comprises classifying the temporal profile.

114. The system of claim 113 wherein classifying the temporal profile comprises determining whether the temporal profile indicates the search query is temporally ambiguous.

115. The system of claim 113 wherein classifying the temporal profile comprises determining whether the temporal profile indicates the search query is at least one of:

(a) temporally ambiguous;

(b) temporally unambiguous;

or atemporal.

116. The system of claim 113 wherein classifying the temporal profile comprises determining whether the temporal profile indicates the search query is not temporally unambiguous;

and not atemporal.

117. The system of claim 102 wherein influencing comprises influencing a rank order of the search results in a listing based on the temporal profile.

118. The system of claim 117 comprising receiving the temporal relevance feedback from a user input to the interactive timeline.

119. The system of claim 118 further comprising generating search results for presentation to the user based on the temporal relevance feedback received from the user.

120. The system of claim 102 wherein identifying the temporal profile further comprises smoothing the temporal data associated with a results set of documents.

121. The system of claim 102 wherein influencing comprises modifying an algorithm for retrieving documents.

122. A system for providing search results in response to a search query, the system comprising:

a) a computer adapted to receive a search query; and

b) the computer comprising a processor, the processor comprising processor executable instructions so as to be capable of;

receiving a search query;

identifying a temporal profile of the search query from temporal data associated with documents retrieved in response to the search query;

determining temporal features of the temporal profile using at least one of;

(a) Kullback-Leibler divergence;

(b) autocorrelation;

(c) kurtosis;

(d) Kleinberg'"'"'s burst model.

predicting an average precision of the search query using a combination of at least one of the temporal features of the temporal profile and a content feature of the documents retrieved in response to the search query.

123. The system of claim 122 wherein predicting comprises using a learning algorithm.

124. The system of claim 123 wherein predicting comprises using at least one of:

(a) a decision tree;

(b) a neural network;

or (c) a support vector machine.

125. The system of claim 122 wherein identifying comprises using a Kullback-Leibler divergence.

126. The system of claim 122 wherein identifying the temporal profile further comprises smoothing the temporal data associated with a results set of documents.