Automated analysis of user search behavior

• US 8,037,042 B2
• Filed: 05/10/2007
• Issued: 10/11/2011
• Est. Priority Date: 05/10/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Robert Ching-Huei, Anderson, Blake Edward
• Primary Examiner(s): Lee; Wilson
• Assistant Examiner(s): Le; Jessica N
• Application Number:
  US11/803,019
• Publication Number:
  US 20080281809A1
• Time in Patent Office: 1,615 Days
• Field of Search: None
• US Class Current: 707/705
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 30/0256 User search
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

Automated analysis of user search behavior is provided. Data on user searches is maintained in a user search database. Relevance factors are determined for each search result included in a given search session where the relevance factors provide an indication of user satisfaction with particular search results included in the session. The relevance factors for each search result are analyzed by a relevance classification module for classifying each search result in terms of its relevance to an associated search query. The result of the relevance classification may assign a relevance classification and associated confidence level to each analyzed search result as to whether the search result is acceptable, unacceptable or partially acceptable relative to the search query that resulted in the search result. Relevance classifications for each analyzed search result may be stored for future use, for example, for diagnostic analysis of the operation of a given search mechanism.

Claims

1. A method of analyzing search behavior, the method comprising:

storing, by one or more computers, data associated with searches and user behavior with regard to a plurality of search results produced by the searches;

identifying, by the one or more computers, a session comprising one or more related search results in the plurality of search results;

prior to calculating one or more relevance factors;

processing, by the one or more computers, the data to determine search queries that were used to locate each search result in the session;

processing, by the one or more computers, the data to determine click-through behavior for each search result in the session; and

processing, by the one or more computers, the data to obtain explicit feedback on search results in the session;

calculating, by the one or more computers, the one or more relevance factors for each search result in the session, wherein a relevance factor for a search result is data that describes an aspect of user behavior regarding the search result, and wherein calculating the one or more relevance factors comprises;

calculating, by the one or more computers, the one or more relevance factors using the search queries that were used to locate each search result in the session;

calculating, by the one or more computers, the one or more relevance factors using the click-through behavior for each search result in the session; and

calculating, by the one or more computers, the one or more relevance factors using the explicit feedback on each search results in the plurality of search results;

determining a first user behavior profile based on one or more relevance factors for search results in the plurality of search results having explicit feedback indicating that the search results are acceptable;

determining a second user behavior profile based on one or more relevance factors for search results in the plurality of search results having explicit feedback indicating that the search results are not acceptable; and

assigning a relevance classification of acceptable to a given search result in the session when the relevance factors for the given search result are similar to the relevance factors for the first user behavior profile, wherein no explicit feedback is provided for the given search result;

assigning, by the one or more computers, a relevance classification of unacceptable to the given search result when the relevance factors for the given search result are similar to the relevance factors for the second user behavior profile.

2. The method of claim 1, wherein identifying the session includes:

identifying, by the one or more computers, one or more search results in the plurality of search results obtained in a prescribed time period.

3. The method of claim 1, further comprising:

determining, by the one or more computers, a third user behavior profile based on one or more relevance factors associated with the search results for which explicit feedback is provided and for which the search results are partially acceptable to the searching user.

4. The method of claim 3, further comprising:

assigning a relevance classification of partially acceptable to the given search result when the relevance factors for the given search result are similar to the relevance factors on which the third user behavior profile is based.

5. The method of claim 1, further comprising:

storing, by the one or more computers, the relevance classification for the given search result; and

performing relevance diagnostics based on the relevance classification assigned to the given search result.

6. A computer system that analyzes search behavior, the computer system comprising:

at least one processing unit; and

one or more memory storage devices storing;

a user search database operative to store data associated with searches and user behavior with regard to a plurality of search results produced by the searches;

a session identifier module that, when executed by the processing unit, is operative to identify a session comprising one or more related search results in the plurality of search results, the related search results produced by a plurality of searches initiated by a user;

a relevance calculation module that, when executed by the processing unit, is operative to;

prior to calculating one or more relevance factors;

process the data to determine search queries that were used to locate each search result in the session;

process the data to determine click-through behavior for each search result in the session; and

process the data to obtain explicit feedback on search results in the session;

calculate the one or more relevance factors for each search result in the session, wherein each of the relevance factors for each of the search results in the session is data that describes an aspect of the user'"'"'s behavior with regard to the search result, and wherein the one or more relevant factors are calculated using the search queries that were used to locate each search result in the session, the click-through behavior for each search result in the session, and the explicit feedback on each search results in the plurality of search results;

determine a first user behavior profile based on one or more relevance factors for search results in the plurality of search results having the explicit feedback indicating that the search results are acceptable;

determine a second user behavior profile based on one or more relevance factors for search results in the plurality of search results having the explicit feedback indicating that the search results are not acceptable, the search queries that were used to locate each search result in the session, and the click-through behavior for each search result in the session; and

a relevance classifier module that, when executed by the processing unit, is operative to;

assign a relevance classification of acceptable to each clicked-through search result in the session for which no explicit feedback is provided and for which the relevance factors for the clicked-through search result are similar to the relevance factors for the first user behavior profile;

assign a relevance classification of unacceptable to each clicked-through search result for which no explicit feedback is provided and for which the relevance factors for the clicked-through search result are similar to the relevance factors for the second user behavior profile.

7. The computer system of claim 6, wherein the session identifier module, when executed by the processing unit, is further operative to identify the session by identifying one or more search results in the plurality of search results obtained in a prescribed time period.

8. The computer system of claim 6,

wherein the relevance classifier module, when executed by the processing unit, is further operative to determine a third user behavior profile based on one or more relevance factors for search results for which explicit feedback is provided and for which the search result is partially acceptable to a searching user;

and

wherein the relevance classifier module, when executed by the processing unit, assigns a relevance classification of partially acceptable to each clicked-through search result in the session for which no explicit feedback is provided and for which the relevance factors for clicked-through search result are similar to the relevance factors for the third user behavior profile.

9. A computer readable storage medium containing computer executable instructions which when executed cause a computer to:

identify a session comprising one or more related search results in a plurality of search results, each search result in the session obtained by a search mechanism from a single user within in a prescribed time period;

prior to calculating one or more relevance factors;

process the data to determine search queries that were used to locate each search result in the session;

process the data to determine click-through behavior for each search result in the session; and

process the data to obtain explicit feedback on search results in the session;

calculate one or more relevance factors for clicked-through search results in the session, wherein for each of the clicked-through search results in the session, each of the relevance factors for the clicked-through search result is data that describes an aspect of the user'"'"'s behavior regarding the clicked-through search result, and wherein the one or more relevance factors are further calculated by using the search queries that were used to locate each search result in the session, the click-through behavior for each search result in the session, and the explicit feedback on each search result in the plurality of search results;

train a SQL decision tree data mining model to;

associate a first user behavior profile with one or more relevance factors associated with search results in the plurality of search results for which explicit feedback is provided and for which the search results are acceptable to a searching user;

associate a second user behavior profile with one or more relevance factors associated with search results for which explicit feedback is provided and for which the search results are not acceptable to the searching user; and

associate a third user behavior profile with one or more relevance factors associated with search results for which explicit feedback is provided and for which the search results are partially acceptable to the searching user;

for each clicked-through search result in the session for which no explicit feedback is provided;

send a Data Mining Extensions (DMX) query to the SQL decision tree data mining model, the DMX query including the relevance factors for the clicked-through search result;

receive, from the SQL decision tree data mining model in response to the DMX query, a relevance classification of acceptable for the clicked-through search result when the relevance factors for the clicked-through search result are similar to the relevance factors associated with the first user behavior profile;

receive, from the SQL decision tree data mining model in response to the DMX query, a relevance classification of unacceptable for the clicked-through search result when the relevance factors for the clicked-through search result are similar to the relevance factors associated with the second user behavior profile;

receive, from the SQL decision tree data mining model in response to the DMX query, a relevance classification of partially acceptable for the clicked-through search result when the relevance factors for the clicked-through search result are similar to the relevance factors associated with the third user behavior profile; and

calculate a confidence level for the clicked-through search result, the confidence level for the clicked-through search result indicating a confidence with which the relevance classification for the clicked-through search result is assigned to the search result;

wherein the relevance classifications assigned to the clicked-through search results in the session are further based on relevance of each of the search results in the session to search queries that were used to locate each of the search results in the session.