Best-bet recommendations

• US 8,082,247 B2
• Filed: 07/30/2009
• Issued: 12/20/2011
• Est. Priority Date: 07/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kumthekar, Aditee, Zhao, Mingyang, Sridharan, Harini, Chang, Qing
• Primary Examiner(s): Lin, Shew-Fen
• Application Number:
  US12/512,845
• Publication Number:
  US 20110029509A1
• Time in Patent Office: 873 Days
• Field of Search: 707/999.005, 707722-757
• US Class Current: 707/727
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

Best-bet recommendation may be provided. User behavior associated with a search result may be logged and applied to a relevance classifier. The relevance classifier may determine whether the user behavior indicates that the search result may comprise a best-bet result. A best-bet result may comprise a result associated with a high confidence that the result may be satisfactory to a user. The relevance classifier may be trained according to past searches and user behavior to improve the confidence on best-bet recommendations. Once confidence in a search result is high enough, future searches on the same query may provide the best-bet result to the searching user.

Claims

1. A method for providing a search recommendation, the method comprising:

training a relevance classifier, wherein the relevance classifier comprises a decision tree comprising a plurality of nodes and wherein training the relevance classifier comprises;

receiving a plurality of log data associated with users'"'"' behavior, wherein each of the plurality of log data comprises a relevance factor and a feedback category comprising at least one of the following categories;

accepted, explored, and rejected,

for each of the plurality of log data, determining whether the associated relevance factor is associated with a node of a decision tree,

in response to determining that the associated relevance factor is not associated with a node of the decision tree, create a new decision node associated with the relevance factor,

in response to determining that the associated relevance factor is associated with a node of the decision tree, for each of the plurality of log data, updating a feedback category count associated with a decision node with the relevance factor of the log data according to the feedback category of the log data, and

for each of the plurality of decision nodes, assigning a confidence value according to the feedback category count of the decision node;

receiving a search query from a user;

providing a plurality of search results to the user;

capturing a behavior associated with the user and at least one of the plurality of search results;

classifying a relevance for the at least one of the plurality of search results according to the captured behavior, wherein the captured behavior comprises at least one of the relevance factors associated with the relevance classifier;

determining, according to the relevance classified for the at least one of the plurality of search results, whether the at least one of the plurality of search results comprises a potential best-bet search result;

in response to determining that the at least one of the plurality of search results comprises the potential best-bet search result, determining whether the confidence value associated with the at least one of the relevance factors is greater than a threshold value; and

in response to determining that the confidence value associated with the at least one of the relevance factors is greater than the threshold value adding the at least one of the plurality of search results to a store comprising recommended search results.

2. The method of claim 1, wherein the captured behavior comprises at least one of the following:

a time the search query was issued, a time the at least one of the plurality of search results was selected, a display position of the at least one of the plurality of search results in a list of the plurality of search results, a dwell time of the at least one of the plurality of search results, a user feedback associated with the at least one of the plurality of search results, an interaction of the user with the at least one of the plurality of search results, and a count of selections by the user from the plurality of search results.

3. The method of claim 1, wherein classifying the relevance for the at least one of the plurality of search results according to the captured behavior comprises classifying the captured behavior according to at least one of the following categories:

accepted, explored, and rejected.

4. The method of claim 1, further comprising evaluating a plurality of captured behaviors associated with the at least one of the plurality of search results.

5. The method of claim 4, wherein each of the plurality of captured behaviors are associated with a confidence.

6. The method of claim 1, further comprising:

receiving a second search query from a second user;

determining whether the store comprising recommended search results comprises a search result associated with the second search query; and

in response to determining that the store comprising recommended search results comprises a best-bet search result associated with the second search query, providing the best-bet search result associated with the second search query to the second user.

7. The method of claim 6, wherein determining whether the at least one of the plurality of search results comprises potential best-bet search result comprises determining whether the captured behavior is associated with a total number of positive feedback references greater than a relevance metric.

8. The method of claim 7, wherein the relevance metric comprises a minimum number of positive feedback references.

9. The method of claim 7, wherein the relevance metric comprises a ratio of positive feedback references to negative feedback references.

10. The method of claim 1, further comprising evaluating a plurality of captured behaviors associated with at least one second result of the plurality of search results.

11. The method of claim 1, wherein classifying a relevance for the at least one of the plurality of search results according to the captured behavior comprises applying a plurality of captured behaviors associated with the at least one of the plurality of search results to a decision tree.

12. The method of claim 11, wherein each of a plurality of nodes associated with the decision tree comprises a confidence value.

13. A computer-readable medium which stores a set of instructions which when executed performs a method for providing best-bet recommendations, the method executed by the set of instructions comprising:

training a relevance classifier, wherein the relevance classifier comprises a decision tree comprising a plurality of nodes and wherein training the relevance classifier comprises;

receiving a plurality of log data associated with users'"'"' behavior, wherein each of the plurality of log data comprises a relevance factor and a feedback category comprising at least one of the following categories;

accepted, explored, and rejected,

for each of the plurality of log data, determining whether the associated relevance factor is associated with a node of a decision tree,

in response to determining that the associated relevance factor is not associated with a node of the decision tree, creating a new decision node associated with the relevance factor,

in response to determining that the associated relevance factor is associated with a node of the decision tree, for each of the plurality of log data, updating a feedback category count associated with a decision node with the relevance factor of the log data according to the feedback category of the log data, and

for each of the plurality of decision nodes, assigning a confidence value according to the feedback category count of the decision node;

providing a plurality of search results in response to receiving a search query;

logging a data element associated with at least one of the plurality of search results, wherein the logged data element comprises a captured user behavior associated with at least one of the relevance factors associated with the relevance classifier;

classifying a relevance for the at least one of the plurality of search results according to an application of the logged data element to the relevance classifier;

determining whether the at least one of the plurality of search results comprises a potential best-bet search result according to the classified relevance;

in response to determining that the at least one of the plurality of search results comprises a potential best-bet search result according to the classified relevance, providing a recommendation that the at least one of the plurality of search results be surfaced as a best-bet search result;

receive the search query from a second user;

determine whether the recommendation that the at least one of the plurality of search results be surfaced as the best-bet search result has been approved; and

in response to determining that the recommendation that the at least one of the plurality of search results be surfaced as the best-bet search result has been approved, provide the best-bet search result to the second user.

14. The computer-readable medium of claim 13, wherein providing the recommendation that the at least one of the plurality of search results be surfaced as a best-bet search result comprises:

notifying an administrator of the recommendation;

determining whether the administrator has approved the potential best-bet search result; and

in response to determining that the administrator has approved the potential best-bet search result, associating the potential best-bet search result with the search query.

15. The computer-readable medium of claim 13, wherein classifying a relevance for the at least one of the plurality of search results according to the logged data element comprises assigning a confidence to the at least one of the plurality of search results according to a decision tree, wherein at least one branch of the decision tree comprises the logged data element.

16. The computer-readable medium of claim 15, wherein each of a plurality of branches of the decision tree comprises a logged data element independent of a content of the search query.

17. The computer-readable medium of claim 15, wherein the decision tree comprises a plurality of nodes each comprising a feedback metric associated with one of a plurality of data elements.

18. The computer-readable medium of claim 17, wherein the feedback metric comprises a count of accepted results, a count of explored results, and a count of rejected results.

19. A system for providing best-bet search recommendations, the system comprising:

a memory storage; and

a processing unit coupled to the memory storage, wherein the processing unit is operative to;

train a relevance classifier, wherein the relevance classifier comprises a decision tree comprising a plurality of nodes and wherein being operative to train the relevance classifier comprises being operative to;

receive a plurality of log data associated with users'"'"' behavior, wherein each of the plurality of log data comprises a relevance factor and a feedback category comprising at least one of the following categories;

accepted, explored, and rejected,

for each of the plurality of log data, determine whether the associated relevance factor is associated with a node of a decision tree,

in response to determining that the associated relevance factor is not associated with a node of the decision tree, create a new decision node associated with the relevance factor,

in response to determining that the associated relevance factor is associated with a node of the decision tree, for each of the plurality of log data, update a feedback category count associated with a decision node with the relevance factor of the log data according to the feedback category of the log data, and

for each of the plurality of decision nodes, assign a confidence value according to the feedback category count of the decision node;

receive a search query from a user;

provide a plurality of search results to the user;

capture a behavior associated with the user and at least one of the plurality of search results, wherein the captured behavior comprises at least one of the relevance factors associated with the relevance classifier;

classify a relevance for the at least one of the plurality of search results according to an application of the captured behavior to the relevance classifier;

determine whether the at least one of the plurality of search results comprises a potential best-bet search result according to the classified relevance;

in response to determining that the at least one of the plurality of search results comprises the potential best-bet search result according to the classified relevance, notify an administrator for a recommendation;

determine whether the administrator has approved the potential best-bet search result;

in response to determining that the administrator has approved the best-bet search result as a best-bet search result, associate the best-bet search result with the search query;

receive the search query from a second user; and

provide the best-bet search result to the second user in response to receiving the search query.