Automated satisfaction measurement for web search

• US 20050125390A1
• Filed: 03/22/2004
• Published: 06/09/2005
• Est. Priority Date: 12/03/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Dumais, Susan T., Hurst-Hiller, Oliver, Watson, Eric
• Granted Patent: US 7,937,340 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 7/00 Methods or arrangements for processing data by operating upon the order or content of the data handled logic circuits H03K19/00

Abstract

Context-based user behavior data is collected from a search mechanism. This data includes, for a given query, user feedback (implicit and explicit) on the query and context information on the query. A predictive pattern is applied to the context-based user behavior data in order to produce predicted user satisfaction data. Data mining techniques may be used to create and improve one or more predictive patterns. Predicted user satisfaction data can be used to monitor or improve search mechanism performance, via a display reporting the performance or identification of any queries with a shared characteristic and sub-par user satisfaction. A dynamically-improving search mechanism uses the predicted user satisfaction data to improve the performance of the search mechanism.

Claims

1. A method for obtaining predicted user satisfaction data regarding the performance of a search mechanism which provides search results in response to user queries, comprising:

storing at least one predictive pattern for predicting user satisfaction with said provided search results from data regarding user behavior in response to a query; and

applying said predictive pattern to at least one element of context-based user behavior data, said elements of context-based user behavior data comprising a performed query;

provided search results; and

user behavior data.

2. The method of claim 1, where said storing at least one predictive pattern comprises utilizing data mining techniques to determine at least one predictive pattern for user satisfaction.

3. The method of claim 1, where said user behavior data comprises explicit user feedback data collected from said user contemporaneously with said performed query.

4. The method of claim 1, where said user behavior data comprises implicit user feedback data.

5. The method of claim 4, where said user behavior data is selected from the group comprising:

user navigation to a new page using a hyperlink;

user navigation to a new page using a history list;

user navigation to a new page using an address bar;

user navigation to a new page using a favorites list;

user scrolling behavior;

user document printing behavior;

user adding a document to said favorites list;

user switching focus to a different application;

user switching focus back from a different application;

user closing a window;

user dwell time behavior;

user initiation of a new query;

sequences of user behaviors; and

user inactivity without switching focus from a window relating to said performed query.

6. The method of claim 1, where said application of said predictive pattern yields predicted user satisfaction data regarding said search mechanism, and where said method further comprises:

displaying said predicted user satisfaction data.

7. The method of claim 1, where said application of said predictive pattern further comprises isolating a set of said performed queries which are unsatisfactory and which share a common characteristic.

8. The method of claim 1, where said context-based user behavior data comprises a testing set of context-based user behavior data.

9. At least one of an operating system, a computer readable medium having stored thereon a plurality of computer-executable instructions, a co-processing device, a computing device, and a modulated data signal carrying computer executable instructions for performing the method of claim 1.

10. A system for obtaining predicted user satisfaction data regarding the performance of a search mechanism which provides search results in response to user queries, comprising:

storage for storing at least one predictive pattern for predicting user satisfaction with a said provided search results from data regarding user behavior in response to a query; and

data mining apparatus for applying said predictive pattern to at least one element of context-based user behavior data, said elements of context-based user behavior data comprising a performed query;

provided search results; and

user behavior data.

11. The system of claim 10, where said predictive pattern is derived from the use of data mining techniques to determine at least one predictive pattern for user satisfaction.

12. The system of claim 10, where said user behavior data comprises explicit user feedback data collected from said user contemporaneously with said performed query.

13. The system of claim 10, where said user behavior data comprises implicit user feedback data.

14. The system of claim 13, where said user behavior data is selected from the group comprising:

user navigation to a new page using a hyperlink;

user navigation to a new page using a history list;

user navigation to a new page using an address bar;

user navigation to a new page using a favorites list;

user scrolling behavior;

user document printing behavior;

user adding a document to said favorites list;

user switching focus to a different application;

user switching focus back from a different application;

user closing a window;

user dwell time behavior;

user initiating a new query;

sequences of user behaviors; and

user inactivity without switching focus from a window relating to said performed query.

15. The system of claim 10, where said data mining apparatus produces predicted user satisfaction data regarding said search mechanism, and where said method further comprises:

displaying said predicted user satisfaction data.

16. The system of claim 10, where said data mining apparatus further isolates a set of said performed queries which are unsatisfactory and which share a common characteristic.

17. The system of claim 10, where said context-based user behavior data comprises a testing set of context-based user behavior data.

18. A method for real-time optimization of a search mechanism which provides search results in response to user queries, comprising:

storing at least one predictive pattern for predicting user satisfaction with a said provided search results from data regarding user behavior in response to a query;

applying said predictive pattern to at least one element of context-based user behavior data, said elements of context-based user behavior data comprising a performed query;

provided search results; and

user behavior data;

modifying said search mechanism based on the result of said application of said predictive pattern.

19. The method of claim 18, where said modification of said search mechanism comprises modifying said search mechanism so said search results for a given query are presented in a different order.

20. The method of claim 18, where said context-based user behavior data comprises a pre-judged set of user behavior data.

21. At least one of an operating system, a computer readable medium having stored thereon a plurality of computer-executable instructions, a co-processing device, a computing device, and a modulated data signal carrying computer executable instructions for performing the method of claim 18.

22. A system for real-time optimization of a search mechanism which provides search results in response to user queries, comprising:

storage for storing at least one predictive pattern for predicting user satisfaction with said provided search results from data regarding user behavior in response to a query;

data mining apparatus for applying said predictive pattern to at least one element of context-based user behavior data, said elements of context-based user behavior data comprising a performed query;

provided search results; and

user behavior data; and

dynamic search mechanism modifier for modifying search mechanism based on the result of said application of said predictive pattern.

23. The system of claim 22, where said dynamic search mechanism modifier modifies said search mechanism so said search results for a given query are presented in a different order.

24. The system of claim 22, where said context-based user behavior data comprises a pre-judged set of user behavior data.

25. A system for real-time optimization of a search mechanism which provides search results in response to user queries, comprising:

means for storing at least one predictive pattern for predicting user satisfaction with said provided search results from data regarding user behavior in response to a query; and

means for applying said predictive pattern to at least one element of context-based user behavior data, said elements of context-based user behavior data comprising a performed query;

provided search results; and

user behavior data.

26. The system of claim 25, further comprising:

means for modifying said search mechanism based on the result of said application of said predictive pattern.