Evaluating recommendations

• US 8,762,310 B2
• Filed: 08/16/2012
• Issued: 06/24/2014
• Est. Priority Date: 03/30/2007
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Whitman, Ronald M., Turner, Wesley M., Yi, Jin Y.
• Primary Examiner(s): VINCENT, DAVID ROBERT
• Application Number:
  US13/587,764
• Publication Number:
  US 20120317067A1
• Time in Patent Office: 677 Days
• Field of Search: 706/12, 706/20, 706/45, 706/62, 705/14.4, 705/14.41
• US Class Current: 706/45
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

Generally described, embodiments of the present disclosure are directed toward the identification of items for inclusion in a recommendations list that may be displayed concurrently with an item selected by a user or users. The recommended items may be items that are of potential interest to the user and/or may be items that are related to the selected item. More specifically, embodiments of the present disclosure provide a method and system for generating one or more recommendations lists, providing those lists to users, evaluating users'"'"' interactions with those lists, and modifying the engines or techniques used to identify items that are to be included in the recommendations lists.

Claims

1. A computer-implemented method for evaluating recommendations, the method comprising:

generating a first set of recommendations including a first plurality of items, wherein the first plurality of items is identified by a first technique using a first set of inputs;

generating a second set of recommendations including a second plurality of items, wherein the second plurality of items is identified by a second technique using a second set of inputs;

determining a first set of end user actions, wherein each of the first set of end user actions corresponds to an action taken by an end user in response to a display corresponding to at least one item of the first set of recommendations;

determining a second set of end user actions, wherein each of the second set of end user actions corresponds to an action taken by an end user in response to a display corresponding to at least one item of the second set of recommendations; and

associating a performance value with the first set of recommendations based at least in part on a comparison of the first set of end user actions and the second set of end user actions.

2. The computer-implemented method of claim 1, wherein the first technique and the second technique are the same.

3. The computer-implemented method of claim 1, wherein the first set of inputs and the second set of inputs are the same.

4. The computer-implemented method of claim 1, wherein each end user action is associated with a specific value, and wherein the performance value of the first set of end user actions is determined based at least in part on the sum of the specific values of each end user action of the first set of end user actions.

5. The computer-implemented method of claim 4, wherein at least one end user action is associated with a negative value.

6. A system for evaluating recommendations, the system comprising:

one or more computing devices configured to;

generate a first set of recommendations including a first plurality of items, wherein the first plurality of items is identified by a first technique using a first set of inputs;

generate a second set of recommendations including a second plurality of items, wherein the second plurality of items is identified by a second technique using a second set of inputs;

determine a first set of end user actions, wherein each of the first set of end user actions corresponds to an action taken by an end user in response to a display corresponding to at least one item of the first set of recommendations;

determine a second set of end user actions, wherein each of the second set of end user actions corresponds to an action taken by an end user in response to a display corresponding to at least one item of the second set of recommendations; and

associate a performance value with at least one of the first set of recommendations or the second set of recommendations based at least in part on a comparison of the first set of end user actions and the second set of end user actions.

7. The system of claim 6, wherein each end user action is associated with a specific value, and wherein the performance value is determined based at least in part on the sum of the specific values of each end user action of at least one of the first set of end user actions or the second set of end user actions.

8. The system of claim 7, wherein at least one end user action is associated with a negative value.

9. The system of claim 6, wherein the one or more computing devices are further configured to perform a responsive action based at least in part on the performance value.

10. The system of claim 9, wherein the responsive action corresponds to a selection of either the first set of recommendations or the second set of recommendations based at least in part on the performance value.

11. The system of claim 9, wherein the responsive action corresponds to modifying at least one characteristic value associated with at least one of the first technique or the second technique.

12. The system of claim 6, wherein the one or more computing devices are further configured to identify an update for at least one of the first technique or the second technique based on the performance value.

13. The system of claim 6, wherein the one or more computing devices are further configured to generate a third set of recommendations comprising one or more items from the first set of recommendations and one or more items from the second set of recommendations.

14. The system of claim 13, wherein each of the first set of end user actions corresponds to an action taken by an end user in response to the third set of recommendations.

15. The system of claim 13, wherein each of the second set of end user actions corresponds to an action taken by an end user in response to the third set of recommendations.

16. A computer-readable, non-transitory storage medium having computer-executable modules for evaluating recommendations, the computer-executable modules comprising:

a recommendation module configured to;

generate a first set of recommendations including a first plurality of items, wherein the first plurality of items is identified by a first technique using a first set of inputs;

generate a second set of recommendations including a second plurality of items, wherein the second plurality of items is identified by a second technique using a second set of inputs;

determine a first set of end user actions, wherein each of the first set of end user actions corresponds to an action taken by an end user in response to a display corresponding to an item from the first set of recommendations;

determine a second set of end user actions, wherein each of the second set of end user actions corresponds to an action taken by an end user in response to a display corresponding to an item from the second set of recommendations; and

associate a performance value with at least one of the first set of recommendations or the second set of recommendations based at least in part on a comparison of the first set of end user actions and the second set of end user actions.

17. The computer-readable, non-transitory storage medium of claim 16, wherein each end user action is associated with a specific value, and wherein the performance value is determined based at least in part on the sum of the specific values of each end user action of the set of end user actions.

18. The computer-readable, non-transitory storage medium of claim 17, wherein at least one end user action is associated with a negative value.

19. The computer-readable, non-transitory storage medium of claim 16, wherein the recommendation module is further configured to perform a responsive action based at least in part on the performance value.

20. The computer-readable, non-transitory storage medium of claim 19, wherein the responsive action corresponds to a selection of either the first set of recommendations or the second set of recommendations based at least in part on the a performance value.

21. The computer-readable, non-transitory storage medium of claim 19, wherein the responsive action corresponds to modifying at least one characteristic value associated with the first technique.

22. The computer-readable, non-transitory storage medium of claim 16, wherein the recommendation module is further configured to generate a third set of recommendations comprising one or more items from the first set of recommendations and one or more items from the second set of recommendations.

23. The computer-readable, non-transitory storage medium of claim 22, wherein each of the first set of end user actions corresponds to an action taken by an end user in response to the third set of recommendations.

24. The computer-readable, non-transitory storage medium of claim 16, wherein the recommendation module is further configured to identify an update for at least one of the first technique or the second technique based on the performance value.