Search system using user behavior data

• US 7,363,282 B2
• Filed: 12/03/2003
• Issued: 04/22/2008
• Est. Priority Date: 12/03/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Shimizu, Takeshi, Wang, Lu, Dumais, Susan T., Zhang, Ying, Karnawat, Kuldeep, Taylor, Thomas A., White, Thomas D., Mydland, Mark B., Fox, Steven C.
• Primary Examiner(s): Vincent; David
• Assistant Examiner(s): FERNANDEZ RIVAS, OMAR F
• Application Number:
  US10/727,444
• Publication Number:
  US 20050125382A1
• Time in Patent Office: 1,602 Days
• Field of Search: 706 1- 18, 706/20, 709/201, 709/227, 709/203, 715/700, 715/701, 715/745, 715/789, 715/780, 715/811, 707/3, 707/5, 707/6, 707/1, 707/10
• US Class Current: 706/45
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9536 Search customisation based on social or collaborative filtering
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

Context-based user feedback is gathered regarding searches performed on a search mechanism. The search mechanism is monitored for user behavior data regarding an interaction of a user with the search mechanism. The response data provided by the search mechanism is also monitored. Context data (describing the search) and user feedback data (the user'"'"'s feedback on the search—either explicit or implicit) are determined. This can be used, for example, to evaluate a search mechanism or to check a relevance model.

Claims

1. A method with regard to a user performing a search during a single search session at a search engine by way of a search mechanism, said method comprising:

monitoring the search mechanism for user behavior data regarding interactions between the user and the search mechanism during the search session thereat, the user behavior data comprising data concerning a plurality of events, each event corresponding to an action of the user at the search mechanism during the search session;

monitoring said search mechanism for response data regarding said search session, the response data comprising a results list;

determining context data describing a search during said search session, the context data being derived from the user behavior data and from the response data and representing an overall context of the search conducted during the search session; and

determining user feedback data describing said search, the user feedback data including implicit user feedback derived from the user behavior data and explicit user feedback regarding the usefulness to a user of the response data and the user response to at least one question, the implicit feedback data comprises time spent reviewing a specific item of the results list, wherein the time spent is calculated by subtracting any time that a user switched to another application while reviewing the specific item, and wherein the explicit user feedback data comprises (a) a user rating of the quality or usefulness of the specific item reviewed from the results list and (b) the user response to the at least one question concerning the results list as a whole; and

performing a context-dependent evaluation of the results of the search engine acquired during the search session, the evaluation based at least in part on the determined context data and the determined user feedback data acquired during the single search session.

2. The method of claim 1, where said search mechanism is a web browser.

3. The method of claim 2, where each action of the user at the search mechanism is selected from among entering a search query;

said user navigation to a new page using a hyperlink;

said user navigation to a new page using a history list;

said user navigation to a new page using an address bar;

said user navigation to a new page using a favorites list;

user scrolling behavior;

user document printing behavior;

said user adding a document to said favorites list;

said user switching focus to a different application;

said user switching focus back from a different application; and

said user closing a window.

4. The method of claim 1 where said method further comprises:

tracking, using a state machine comprising at least two states describing progress through said search, which of said states said search is in.

5. The method of claim 4, where said context data describing said search comprises state data regarding which of said states were tracked during said search.

6. The method of claim 4 where at least one transition between said states in said state machines is at least partially dependent on explicit user feedback.

7. The method of claim 6 where said context data describing said search comprises said explicit user feedback.

8. The method of claim 1 where said context data describing said search comprises user behavior data.

9. The method of claim 1 where said user feedback data comprises explicit user feedback.

10. The method of claim 1 where said user feedback data comprises implicit user feedback based on said user behavior data.

11. A computer-readable medium having computer-executable instructions to perform the method of claim 1.

12. The method of claim 1 wherein the search comprises a number of queries from the user to the search engine, each query being followed by a response from the search engine, the method comprising determining context data that describes each query of the search, including timing and how the user reacted to the corresponding response, and performing the context-dependent evaluation of the results of the search engine based on such context data that allows corresponding user feedback data to be analyzed in a context of the search that is performed during the search session.

13. The method of claim 1 comprising determining user feedback data that describes the search, the user feedback data including implicit user feedback derived from user behavior including browsing, scrolling, and clicking behavior.

14. The method of claim 1 comprising determining user feedback data that describes the search, the user feedback data including implicit user feedback including:

user behavior while visiting a result list page, including time spent thereat;

user behavior while exploring a hyperlink on the result list page, including time spent thereat;

user behavior while visiting a result item page, including the time spent thereat; and

user behavior relating to the user ignoring a result item of a result list.

15. The method of claim 1 comprising determining user feedback data that describes the search, the user feedback data including explicit user feedback by way of a dialog box opened at the search mechanism of the user.