Web site system with automated processes for detecting failure events and for selecting failure events for which to request user feedback

• US 7,552,365 B1
• Filed: 10/06/2004
• Issued: 06/23/2009
• Est. Priority Date: 05/26/2004
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Marsh, Brian D., Bezos, Jeffrey P., Hyman, Michael I.
• Primary Examiner(s): Baderman; Scott T.
• Assistant Examiner(s): Truong; Loan
• Application Number:
  US10/959,239
• Time in Patent Office: 1,721 Days
• Field of Search: 714/47, 707/3, 707/5, 715/701
• US Class Current: 714/47.2
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06

Abstract

A web site or other interactive service maintains user activity data reflective of the activities of its users. A failure analysis component analyzes the user activity data to automatically detect likely failure events. A failure event filtering component intelligently selects failure events for which to request feedback from the user, preferably taking into consideration information about the failure event itself and information about the particular user. Requests for user feedback, and operator responses to the user feedback messages, are preferably presented on a personal log page that also displays other types of personalized information.

Claims

1. A computer-implemented method of detecting and processing failure events associated with an interactive system, the method comprising:

monitoring activity of a user to generate user activity data reflective of events that occur during interaction with an interactive system;

programmatically analyzing the user activity data to detect a pattern of user activity indicative of a failure event; and

programmatically evaluating, by execution of code by one or more computers, whether feedback should be requested from the user regarding the failure event such that a decision of whether to request feedback from the user takes into consideration information about the failure event, and further takes into consideration a current number of user feedback messages queued for review by at least one human operator for generating replies to the user feedback messages.

2. The method of claim 1, wherein programmatically analyzing the user activity data comprises analyzing a clickstream history of the user to assess whether the user has failed to achieve a desired objective.

3. The method of claim 1, wherein programmatically analyzing the user activity data comprises analyzing a search history of the user to assess whether the user has failed to locate desired information.

4. The method of claim 3, wherein the search history is analyzed to detect a pattern of related, unsuccessful search attempts.

5. The method of claim 1, wherein programmatically analyzing the user activity data comprises analyzing the contents of a user'"'"'s shopping cart.

6. The method of claim 5, wherein the contents of the user'"'"'s shopping cart are analyzed to determine if the user failed to purchase items added to the shopping cart.

7. The method of claim 1, wherein programmatically evaluating whether feedback should be requested comprises generating a score value that reflects the information about the failure event.

8. The method of claim 1, wherein the decision of whether to request feedback from the user additionally takes into consideration information about the user.

9. The method of claim 8, wherein programmatically evaluating whether feedback should be requested comprises generating a score value that reflects the information about the failure event and the information about the user.

10. The method of claim 8, wherein the information about the user comprises a transaction history of the user, said transaction history being separate from user activity data relied on to detect the failure event.

11. The method of claim 8, wherein the information about the user comprises data indicative of a usefulness of feedback previously provided by the user.

12. The method of claim 8, wherein the information about the user comprises reputation data derived at least in-part from votes cast by other users.

13. The method of claim 1, wherein the information about the user comprises data indicative of the frequency with which the user has previously responded to feedback requests.

14. The method of claim 1, wherein the information about the failure event comprises a type of the failure event.

15. The method of claim 1, wherein programmatically evaluating whether feedback should be requested from the user additionally comprises taking into consideration a current availability of resources for responding to feedback messages from users.

16. The method of claim 1, further comprising, in response to a decision to request feedback from the user, providing a feedback request message to the user that describes the failure event and requests feedback thereon, and providing at least one text field for the user to enter free-form feedback text.

17. The method of claim 16, wherein the feedback request message is a personalized web page presented to the user.

18. The method of claim 17, wherein the personalized web page is a personal log page.

19. The method of claim 16, wherein the feedback request message is an email, instant message, or a pop-up window.

20. The method of claim 16, further comprising, in response to receiving a feedback message from the user regarding the failure event, programmatically analyzing text of the feedback message to categorize the feedback message, said text entered by the user.

21. The method of claim 1, further comprising posting replies to particular user feedback messages for viewing by corresponding users.

22. The method of claim 21, wherein at least some of the replies are generated by a human operator.

23. The method of claim 21, wherein at least some of the replies are generated by an automated system.

24. A system for detecting and processing failure events associated with an interactive service, the system comprising:

a data repository that stores user activity data reflective of events that occur during sessions of users with an interactive service;

a failure analysis agent that analyzes at least the user activity data of the users to detect failure events;

a failure event filtering component that evaluates whether a user that experienced a failure event of a particular type should be contacted to request feedback regarding the failure event, such that a decision whether to request feedback from the user on the failure event is based at least partly on a quantity of feedback data collected from users regarding said particular type of failure event;

a user messaging component that requests feedback from users on selected failure events evaluated by the failure event filtering component; and

one or more computers, wherein the failure analysis agent, failure event filtering component and user messaging component comprise software executed by said one or more computers.

25. The system of claim 24, wherein the failure analysis agent applies a set of rules to determine whether a pattern of events constitutes a failure event, at least some of the rules adapted to detect patterns of user activity indicative of a user failure to achieve a desired objective.

26. The system of claim 24, wherein the failure analysis agent detects failure events in which users are deemed to have conducted unsuccessful searches.

27. The system of claim 24, wherein the failure analysis agent detects failure events in which users have failed to purchase items added to a shopping cart.

28. The system of claim 24, wherein the failure event filtering component additionally takes into consideration a reputation level of the user in evaluating whether the user should be contacted to request feedback regarding the failure event.

29. The system of claim 24, wherein the failure event filtering component additionally takes into consideration, in evaluating whether the user should be contacted, a number of user feedback messages queued for review by at least one human operator for responding to the user feedback messages.

30. The system of claim 24, wherein the failure event filtering component additionally takes into consideration, in evaluating whether feedback should be requested from the user, a level of usefulness of feedback previously provided by the user.

31. The system of claim 24, wherein the failure event filtering component additionally takes into consideration, in evaluating whether feedback should be requested from the user, a transaction history of the user, said transaction history being separate from user activity data that gave rise to the failure event.

32. The system of claim 24, wherein the failure event filtering component additionally takes into consideration an availability of a resource for responding to feedback messages received from users, such that a quantity of feedback request messages sent to users is adjusted adaptively over time to regulate use of said resource.

33. The system of claim 32, wherein the resource is one or more human operators tasked with responding to feedback messages from users.

34. The system of claim 24, further comprising a component that automatically analyzes text of feedback messages received from the users to categorize the feedback messages, said text including free-form text entered by the users.

35. The system of claim 24, wherein the user messaging component presents feedback request messages to users within personalized logs of the users.

36. The system of claim 24, wherein the user messaging component presents feedback request messages to users within an email, instant message, or a pop-up window.

58. The system of claim 24, wherein the user messaging component additionally outputs feedback reply messages to users who submit feedback messages.

37. A method of detecting and processing web site failure events, the method comprising:

monitoring activity of a user to generate user activity data reflective of events that occur during one or more interactive sessions of the user with the web site;

programmatically analyzing the user activity data according to a set of rules to detect a failure event;

generating a score that represents a desirability of contacting the user to request feedback about the failure event, wherein the score reflects information about the failure event and information about the user, and additionally reflects a quantity of user feedback messages queued for human review for generating replies to corresponding users;

determining, based at least in part on the score, whether to request feedback from the user about the failure event; and

if a decision is made to request feedback from the user, sending a request for feedback to the user;

wherein the method is performed by a system that comprises one or more computers.

38. The method of claim 37, wherein the request for feedback is provided in conjunction with an electronic form that is adapted to be completed and submitted by the user to provide feedback regarding the failure event, said electronic form including at least one field for user entry of a free-form text message.

39. The method of claim 37, wherein the score reflects a usefulness of feedback previously provided by the user.

40. The method of claim 37, wherein the step of determining whether to request feedback comprises taking into consideration a frequency with which the user has previously responded to feedback requests.

41. The method of claim 37, wherein the score reflects a previously-derived reputation of a user.

42. The method of claim 37, wherein the score reflects a transaction history of the user, said transaction history being separate from user activity data relied on to detect the failure event.

43. The method of claim 37, wherein the failure event is an event in which the user conducts multiple unsuccessful searches.

44. The method of claim 43, wherein the failure event is detected at least in-part by detecting a search query submission for which no matching items were found.

45. The method of claim 37, wherein the failure event is an event in which the user has failed to purchase items that the user added to a shopping cart.

46. The method of claim 37, wherein the score reflects an availability of a resource for responding to feedback messages from users, whereby a quantity of feedback request messages sent to users is adjusted adaptively over time to regulate resource use.

47. The method of claim 46, wherein the resource is one or more human operators.

48. The method of claim 37, wherein the step of determining whether to request feedback is based in part on a quantity of user feedback collected regarding a type of the failure event.

49. A computer-implemented method of detecting and processing failure events associated with an interactive system, the method comprising:

monitoring activity of a user to generate user activity data reflective of events that occur during interaction with an interactive system;

programmatically analyzing the user activity data to detect a pattern of user activity indicative of a failure event of a particular type; and

programmatically evaluating, via execution of instructions by a system that comprises one or more computers, whether feedback should be requested from the user regarding the failure event such that a decision of whether to request feedback from the user is based at least partly on how much feedback has been collected for said type of failure event.

50. The method of claim 49, wherein the decision of whether to request feedback from the user additionally takes into consideration a transaction history of the user, said transaction history being separate from user activity data relied on to detect the failure event.

51. The method of claim 49, wherein the decision of whether to request feedback from the user additionally takes into consideration data regarding a usefulness of feedback previously provided by the user.

52. The method of claim 49, wherein the decision of whether to request feedback from the user additionally takes into consideration reputation data reflective of a reputation of said user, said reputation data derived, at least in-part, from votes cast by other users.

53. The method of claim 49, wherein programmatically evaluating whether feedback should be requested from the user comprises taking into consideration a quantity of user feedback messages currently queued for review by at least one human operator for purposes of generating replies to the user feedback messages.

54. The method of claim 49, further comprising, in response to a decision to request feedback from the user, providing a feedback request message to the user that describes the failure event and requests feedback thereon, and providing the user access to a feedback form that includes at least one field for user entry of free-form feedback text.

57. The method of claim 49, wherein the decision of whether to request feedback from the user is additionally based on an extent to which the user has previously responded to requests for feedback on detected failure events.

55. A failure event detection and processing system, said system comprising:

one or more computers programmed to detect and process failure events by at least;

monitoring activity of a user to generate user activity data reflective of events that occur during interaction with an interactive system;

analyzing the user activity data to detect a pattern of user activity indicative of a failure event of a particular type; and

evaluating whether feedback should be requested from the user regarding the failure event such that a decision of whether to request feedback from the user is based at least partly on how much feedback has been collected for said type of failure event.

56. Physical computer storage having executable instructions stored thereon, said executable instructions capable of causing a computer system to perform a method that comprises:

monitoring activity of a user to generate user activity data reflective of events that occur during interaction with an interactive system;

programmatically analyzing the user activity data to detect a pattern of user activity indicative of a failure event of a particular type; and

programmatically evaluating whether feedback should be requested from the user regarding the failure event such that a decision of whether to request feedback from the user is based at least partly on how much feedback has been collected for said type of failure event.