Ranking contextual signals for search personalization

• US 8,706,725 B2
• Filed: 03/14/2011
• Issued: 04/22/2014
• Est. Priority Date: 03/14/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bailey, Peter R., Crook, Aidan, Ramsey, William D., Craswell, Nick, Chica, Sebastian De La, Dandekar, Nikhil B., Schwartz, Daniel R.
• Primary Examiner(s): To, Baoquoc N
• Assistant Examiner(s): LODHI, ANDALIB FT
• Application Number:
  US13/047,466
• Publication Number:
  US 20120239646A1
• Time in Patent Office: 1,135 Days
• Field of Search: 707/733, 707/748, 707/769
• US Class Current: 707/733
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence

Abstract

Methods are provided for re-ranking documents based on user-specific features. Search results are received from a non-contextual ranking system such that the search results are not specific toward a particular user, such as the user who submitted the search query. Contextual signals are received and provide user-specific features that are used to re-rank documents so that the most important and relevant documents are listed at the top of the list of search results. Each of the user-specific features are evaluated and compared to determine a new position of each document. A set of contextual search results is then generated based on the new positions.

Claims

1. One or more computer-storage media devices storing computer-useable instructions that, when used by a computing device, cause the computing device to perform a method for using multiple contextual signals to re-rank search results, the method comprising:

receiving a set of non-contextual search results generated as a result of a search query entered by a user;

receiving a set of contextual signals that include user-specific features that can be used to re-rank documents in the set of non-contextual search results, the set of contextual signals being received from a source other than the user;

prior to communicating the set of non-contextual search results for presentation to the user, evaluating each of the user-specific features by a machine-learning model to establish an importance or relevance of each of the user-specific features in relation to the user and the search query;

comparing the importance or relevance of the user-specific features, by the machine-learning model, with a current position of each document in the set of non-contextual search results, wherein the machine-learning model uses one or more algorithms to learn which of the user-specific features are more important in re-ranking the documents in the set of non-contextual search results;

based on the comparison, algorithmically determining a new position of each document in the set of non-contextual search results; and

utilizing the new position of each document in the list of search results to generate a set of contextual search results.

2. The computer-storage media devices of claim 1, wherein the contextual signals are received from two or more contextual signal providers.

3. The one or more computer-storage media devices of claim 1, wherein evaluating each of the user-specific features further comprises utilizing feedback from the user in relation to the set of contextual search results returned to the user.

4. The one or more computer-storage media devices of claim 1, further comprising determining whether each of the contextual signals is below a minimum trusted threshold for the particular user and the received search query.

5. The one or more computer-storage media devices of claim 4, wherein if all of the contextual signals are below the minimum trusted threshold, returning the set of non-contextual search results to the user.

6. The one or more computer-storage media devices of claim 1, wherein the contextual signals further include a confidence score that indicates a confidence level as to how confident a contextual signal provider is with the corresponding user-specific feature.

7. The one or more computer-storage media devices of claim 6, further comprising utilizing the confidence scores to re-rank the documents in the set of non-contextual search results.

8. The one or more computer-storage media devices of claim 1, further comprising:

receiving scores from contextual signal providers indicating a relevancy or an importance of each of the user-specific features;

normalizing the scores by converting the scores into a common numerical space to generate normalized scores; and

adjusting the normalized scores based on one or more of,

a level of trustworthiness associated with a contextual signal provider that provided a particular user-specific feature, or a position bias effect that indicates an effect of positioning a particular document in a certain position.

9. The one or more computer-storage media devices of claim 8, wherein normalizing the scores further comprises generating a probability and a confidence that the user will select a particular document when it is positioned at the top of the set of contextual search results.

10. One or more computer-storage media devices storing computer-useable instructions that, when used by a computing device, cause the computing device to perform a method for using multiple contextual signals to re-rank search results, the method comprising:

receiving a set of non-contextual search results based on a search query received from a user;

receiving contextual signals that provide user-specific features associated with the user, the contextual signals being received from a source other than the user;

prior to communicating the set of non-contextual search results for presentation to the user, utilizing a machine-learning model that uses an algorithm to compare each of the user-specific features associated with the user to determine which of the user-specific features are more important in re-ranking documents in the set of non-contextual search results;

re-ranking the documents in the set of non-contextual search results based on the comparison of the user-specific features to generate a set of contextual search results;

receiving feedback from the user regarding the documents in the set of contextual search results; and

communicating the feedback to the machine-learning model, wherein the machine-learning model utilizes the feedback in its comparison of each of the user-specific features in determining the importance and the relevance of the user-specific features in relation to future search queries received from the user and other users.

11. The one or more computer-storage media devices of claim 10, wherein the feedback from the user comprises selections made by the user of the documents in the set of contextual search results.

12. The one or more computer-storage media devices of claim 10, wherein the feedback indicates the user'"'"'s preference for certain data sources over others, wherein the data sources include web documents, images, weather information, stock information, news articles, and videos.

13. The one or more computer-storage media devices of claim 10, further comprising communicating the set of contextual search results to the user.

14. The one or more computer-storage media devices of claim 10, wherein the contextual signals include two or more of the user'"'"'s location, a location associated with the documents in the set of non-contextual search results, tasks that the user is engaged in, query log data, demographics associated with the user, or social-based data.

15. The one or more computer-storage media devices of claim 10, wherein the machine-learning model normalizes scores from contextual signal providers indicating a relevancy or an importance of each of the user-specific features.

16. The one or more computer-storage media devices of claim 15, further comprising adjusting the normalized scores based on one or more of,

a level of trustworthiness associated with a contextual signal provider that provided a particular user-specific feature, or

a position bias effect that indicates an effect of positioning a particular document in a certain position.

17. One or more computer-storage media devices storing computer-useable instructions that, when used by a computing device, cause the computing device to perform a method for using multiple contextual signals to re-rank search results, the method comprising:

receiving a set of non-contextual search results based on a search query received from a user;

receiving a plurality of contextual signals that provide user-specific features that are used to re-rank documents in the set of non-contextual search results, the plurality of contextual signals including a score indicating a relevancy of each of the user-specific features;

determining that each of the plurality of contextual signals meets a minimum threshold for being trustworthy in relation to the user and the search query;

converting the scores associated with the user-specific features into a common numerical space to generate normalized scores for the user-specific features;

adjusting the normalized scores based on one or more of,

a level of trustworthiness associated with a contextual signal provider that provided a particular user-specific feature, or

a position bias effect that indicates an effect of positioning a particular document in a certain position; and

re-ranking documents in the set of non-contextual search results based on the adjusted normalized scores, wherein normalizing the scores further comprises generating a probability and a confidence that the user will select a particular document when it is positioned at top of a set of contextual search results.

18. The one or more computer-storage media devices of claim 17, further comprising:

generating a probability and a confidence that the user will select a particular document when it is positioned at the top of the set of contextual search results; and

utilizing the probability and the confidence to convert the scores to normalized scores.