Search system using user behavior data

• US 7,590,619 B2
• Filed: 03/22/2004
• Issued: 09/15/2009
• Est. Priority Date: 03/22/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Dumais, Susan T., Hurst-Hiller, Oliver
• Primary Examiner(s): Mofiz; Apu M
• Assistant Examiner(s): STACE, BRENT S
• Application Number:
  US10/805,706
• Publication Number:
  US 20050210024A1
• Time in Patent Office: 2,003 Days
• Field of Search: 707/1, 707/5, 707/100, 707/101, 707/2, 707/3
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

Context-based user behavior data is collected from a search mechanism. This data includes, for a given query, user feedback (implicit and explicit) on the query and context information on the query. This information can be used, for example, to evaluate a search mechanism or to check a relevance model. This context-based user behavior data may include user information. In one embodiment, explicit feedback is requested from the user except when the user requests a pause in explicit feedback requests, or only periodically, in order to reach a target value for requests for explicit feedback. The explicit feedback may include feedback concerning results not visited, and concerning non-standard results. Implicit feedback will include particular data items such as requeries by a user.

Claims

1. A method for improving performance of a search mechanism, which incorporates a browser for user interaction, comprising:

collecting user information;

detecting an event relating to an interaction of the user with the browser;

providing the event as input to a state machine, the state machine comprising a plurality of states, each state corresponding to a state of user interaction with the browser and a state transition function, the state transition function mapping a first state and the event to a second state, whereupon the event causes a transition of the state machine from a current state to a new state based upon the event and the state transition function;

determining context information based upon the new state of the state machine;

determining implicit user feedback data based upon the transition from the current state to the new state, wherein the implicit user feedback data reflects user behavior during a search and includes user behavior while visiting a result list page, user behavior while exploring a hyperlinik on a result list page, user behavior for visiting a result item page or result item ignore behavior;

collecting explicit user feedback data by;

identifying at least one non-selected search result that is generated by the search mechanism as part of said search but that is not selected by the user; and

submitting one or more questions to the user regarding the non-selected search result and receiving explicit user feedback data to said questions, said questions prompting the user for explicit reasons why a non-selected search result failed to correspond to a search request;

utilizing the implicit user feedback data and the explicit user feedback data in light of the context information to identify a problem with the search mechanism; and

correcting the problem to improve performance of the search mechanism.

2. The method of claim 1, where said user information comprises one or more of the following:

a speed of said user'"'"'s connection to said search mechanism;

a type of said user'"'"'s connection to said search mechanism;

a classification of said user'"'"'s use of said search mechanism;

background information concerning said user;

or

a language which said user is using to perform said search.

3. The method of claim 1, where said step of collecting said user information comprises:

requesting said user information from said user; and

accepting responses from said user.

4. A method for improving performance of a search mechanism, wherein the search mechanism includes a browser based on context-based user feedback data, said method comprising:

monitoring of said search mechanism for user behavior data regarding an interaction of a user with the browser to detect an event;

providing the event as input to a state machine, the state machine comprising a plurality of states, each state corresponding to a state of user interaction with the browser and a state transition function, the state transition function mapping a first state and the event to a second state, whereupon the event causes a transition of the state machine from a current state to a new state based upon the event and the state transition function;

determining context information based upon the new state of the state machine;

determining implicit user feedback data based upon the transition from the current state to the new state, wherein the implicit user feedback data reflects user behavior during a search and includes user behavior while visiting a result list page, user behavior while exploring a hyperlinik on a result list page, user behavior for visiting a result item page or result item ignore behavior;

determining if a snooze request specifying a time period to suspend collection of explicit feedback data is in effect from said user, and, if not, collecting explicit feedback data from the user;

identifying at least one non-selected search result that is generated by the search mechanism as part of said search but that is not selected by the user;

acquiring explicit user feedback data describing said search by submitting one or more questions to the user regarding explicit reasons why the non-selected search result failed to correspond to the search, said explicit user feedback data comprising information regarding an extent to which a search result corresponds to a search request;

using the implicit user feedback data and the explicit user feedback data in light of the context information to identify a problem with the search mechanism; and

correcting the problem to improve performance of the search mechanism.

5. The method of claim 4, where said step of determining if a snooze request is in effect from said user comprises:

determining if said user has issued a snooze request; and

determining if an associated time period associated with said snooze request has elapsed.

6. The method of claim 4, further comprising:

storing target data concerning a target value for how often explicit feedback should be collected for searches; and

allowing explicit feedback to be collected only if collecting the explicit feedback would not result in exceeding said target value for how often explicit feedback is collected.

7. A method for improving performance of a search mechanism based on context-based user feedback data, said method comprising:

detecting an event relating to an interaction of the user and a browser;

providing the event as input to a state machine, the state machine comprising a plurality of states, each state corresponding to a state of user interaction with the browser and a state transition function, the state transition function mapping a first state and the event to a second state, whereupon the event causes a transition of the state machine from a current state to a new state based upon the event and the state transition function;

determining context information based upon the new state of the state machine;

determining implicit user feedback data based upon the transition from the current state to the new state, wherein the implicit user feedback data reflects user behavior during a search and includes user behavior while visiting a result list page, user behavior while exploring a hyperlinik on a result list page, user behavior for visiting a result item page or result item ignore behavior;

identifying at least one non-selected search result that is generated by the search mechanism as part of said search but that is not selected by the user;

acquiring explicit user feedback data by submitting one or more questions to the user regarding why the non-selected search result failed to correspond to the search and receiving responses to said questions;

using the implicit user feedback data and the explicit user feedback data in light of the context information to identify a problem with the search mechanism; and

correcting the problem to improve performance of the search mechanism.

8. A system for improving performance of a search mechanism based on context-based user feedback data, said system comprising:

a processor;

a memory having stored therein computer executable instructions;

a user information collector for collecting user information from a user having access to said search mechanism;

a user behavior monitor for monitoring of said search mechanism for raw user behavior data regarding an interaction of said user with said search mechanism to perform a search;

a state machine for conversion of the raw user behavior data into interpreted user behavior data and for generating context information, wherein the context information corresponds to a current state of the state machine;

an explicit user feedback data accumulator for identifying at least one non-selected search result that is generated by the search mechanism as part of said search but that is not selected by the user, the explicit user feedback data accumulator further for acquiring explicit user feedback data describing said search by submitting one or more questions to said user regarding explicit reasons why the non-selected search result failed to correspond to the search and receiving responses to said questions, said questions prompting said user for information regarding an extent to which a search result corresponds to a search request; and

an analysis component for using the explicit user feedback data in light of context data to identify a problem with the search mechanism and to improve the performance of the search mechanism by correcting the problem.

9. The system of claim 8, where said user information comprises one or more of the following:

a speed of said user'"'"'s connection to said search mechanism;

a type of said user'"'"'s connection to said search mechanism;

a classification of said user'"'"'s use of said search mechanism;

background information concerning said user;

or

a language which said user is using to perform said search.

10. The system of claim 8, where said user information collector requests said user information from said user and accepts responses from said user.

11. A system for improving performance of a search mechanism based on context-based user feedback data, said system comprising:

a processor;

a memory having stored therein computer executable instructions;

user behavior monitor for monitoring of said search mechanism for user behavior data regarding an interaction of a user having access to said search mechanism with said search mechanism to perform a search;

context monitor for monitoring said search mechanism for search mechanism response data regarding said search;

explicit feedback collection mechanism for making a determination of whether a snooze request specifying a time period to suspend collection of explicit feedback data is in effect from said user, and, if not, collecting explicit feedback data from said user;

an explicit user feedback data accumulator for identifying at least one non-selected search result that is generated by the search mechanism as part of said search but that is not selected by the user, the explicit user feedback data accumulator further for acquiring explicit user feedback data describing said search by submitting one or more questions to the user regarding the non-selected search result, said explicit user feedback data comprising information regarding explicit reasons why a search result failed to corresponds to a search request; and

an analysis component for using the explicit user feedback data in light of context data to identify a problem with the search mechanism and to improve the performance of the search mechanism by correcting the problem.

12. The system of claim 11, where the determination of whether a snooze request is in effect from said specific user comprises:

a determination of whether said user has issued a snooze request; and

a determination of whether an associated time period associated with said snooze request has elapsed.

13. The system of claim 11, where said explicit feedback collection mechanism stores target data concerning a target value for how often explicit feedback should be collected for searches;

and allows explicit feedback to be collected only if collecting the explicit feedback would not result in exceeding said target value for how often explicit feedback is collected.

14. A system for improving performance of a search mechanism based on context-based user feedback data, said system comprising:

a processor;

a memory having stored therein computer executable instructions;

user behavior monitor for monitoring of said search mechanism for user behavior data regarding an interaction of a user having access to said search mechanism with said search mechanism to perform a search, said user behavior data comprising data concerning requery performed by said user, dwell time on a results page, click time on said results page, position of result clicked, more results requested by said user, result dwell time, result page size, or result page actions;

a context monitor for monitoring said search mechanism wherein the context monitor comprises a state machine, the state machine comprising a plurality of states, each state corresponding to a state of user interaction with the search mechanism and a state transition function, the state transition function mapping a first state and an event to a second state, whereupon an event generated by user interaction with the search mechanism causes a transition of the state machine from a current state to a new state based upon the event and the state transition function, wherein the context monitor determines context information based upon the new state of the state machine;

an explicit user feedback data accumulator for identifying at least one non-selected search result that is generated by the search mechanism as part of said search but that is not selected by the user, the explicit user feedback data accumulator further for acquiring explicit user feedback data describing said search by submitting one or more questions to said user regarding explicit reasons why the non-selected search result failed to correspond to the search and receiving responses to said questions; and

an analysis component for using the explicit user feedback data in light of the context data to identify a problem with the search mechanism and to improve the performance of the search mechanism by correcting the problem.