Managing content based on reputation

• US 8,554,601 B1
• Filed: 08/22/2003
• Issued: 10/08/2013
• Est. Priority Date: 08/22/2003
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Marsh, Brian David, Gloy, Nikolas Clemens
• Primary Examiner(s): Sterrett, Jonathan G
• Application Number:
  US10/646,341
• Time in Patent Office: 3,700 Days
• Field of Search: 705/10
• US Class Current: 705/7.32
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0278 Product appraisal

Abstract

Techniques are described for managing various types of content in order to identify content having attributes of interest, such as based at least in part on automatically generated assessments of the reputations of authors of the content and/or of evaluators of the content. When content of sufficient interest is identified, it can then be provided to readers in various ways. In some situations, the content being evaluated and managed may include item reviews and/or how-to guides provided to a Web merchant by user authors, and in some situations may include information from blogs. Various types of awards can also be provided to authors and/or evaluators based on one or more of multiple types of reputation scores from their assessed reputations. This abstract is provided to comply with rules requiring it, and is submitted with the intention that it not reflect the limit of the scope of the claims.

Claims

1. A computer-implemented method for selecting information to provide to users based on reputations of evaluators of the information, the method comprising:

receiving from a reviewer user a review related to an item available from a Web merchant, the receiving of the review being performed by one or more programmed computing systems of the Web merchant;

receiving multiple evaluations of the review, each of the multiple evaluations being from one of multiple evaluator users who each has an existing reputation weight for the Web merchant that is based at least in part on previous evaluations supplied by that evaluator user for multiple other reviews for items available from the Web merchant, each received evaluation including a quantitative assessment of contents of the review for each of one or more of multiple content rating dimensions available for use in assessing the review;

automatically generating an aggregate assessment of the content of the review based at least in part on combining quantitative assessments from the received evaluations for the review, the generated aggregate assessment being further based on the existing reputation weights of the evaluator users in such a manner that a first quantitative assessment from a first evaluator user with a first reputation weight has a different impact on that generated aggregate assessment than that first quantitative assessment from a distinct second evaluator user with a distinct second reputation weight, the automatic generating being performed by the one or more programmed computing systems;

automatically updating the existing reputation weights for each of one or more of the evaluator users based on a relationship of the quantitative assessments from the evaluation of that evaluator user to the quantitative assessments from the evaluations of other of the evaluator users, the automatic updating being performed by the one or more programmed computing systems; and

for each of multiple additional users of the Web merchant who are distinct from the multiple evaluator users and from the reviewer user, determining whether to provide the review to the additional user based at least in part on the automatically generated aggregate assessment for the content of the review.

2. The method of claim 1 including, before the automatic updating of the reputation weights of the one or more evaluator users, determining whether the received evaluations satisfy an evaluator reputation calculation threshold, and wherein the automatic updating of the reputation weights of the one or more evaluator users is performed only when it is determined that the received evaluations satisfy the evaluator reputation calculation threshold.

3. The method of claim 2 wherein the evaluator reputation calculation threshold is based at least in part on a minimum degree of consensus existing among the received evaluations, and wherein the determining includes automatically calculating the existing degree of consensus among the received evaluations.

4. The method of claim 1 wherein the relationship of the quantitative assessments from the evaluation of an evaluator user to the quantitative assessments from the evaluations of other of the evaluator users that is used when automatically updating the reputation weight for that evaluator user is based on a degree of agreement between the quantitative assessments from the evaluation of the evaluator user and quantitative assessments from a consensus evaluation for the received evaluations.

5. The method of claim 1 wherein the reputation weights of the evaluator users that are used in the automatic generating of the aggregate assessment of the content of the review were automatically generated based on the previous evaluations by those evaluator users.

6. The method of claim 1 including, after the receiving of the evaluations from the evaluator users, for each of at least some of the evaluations receiving one or more ratings of the evaluation from users other than the evaluator user that provided the evaluation, and automatically modifying the reputation weights for evaluator users whose evaluations received ratings based at least in part on those ratings.

7. The method of claim 1 including, after the automatic updating of the reputation weights of the one or more evaluator users, receiving an indication that the content is no longer in use for determining reputation weights of the evaluator users, and automatically updating the reputation weights for each of those evaluator users to remove influence based on the relationship of the quantitative assessments from the evaluation of that evaluator user to the quantitative assessments from the evaluations of other of the evaluator users.

8. The method of claim 1 wherein the automatic generating of the aggregate assessment of the content of the review is further based in part on an existing reputation weight of the reviewer user from which the review was received.

9. The method of claim 8 wherein the reputation weight of the reviewer user is based on a degree of consistency between the automatically generated aggregate assessment of the content of the review and automatically generated aggregate assessments of the content of previous reviews received from the reviewer user.

10. The method of claim 1 including automatically updating a reputation weight of the reviewer user from which the review was received based at least in part on the automatically generated aggregate assessment of the content of the review.

11. The method of claim 10 including, before the automatic updating of the reputation weight of the reviewer user, determining whether the received evaluations satisfy an author reputation calculation threshold, and wherein the automatic updating of the reputation weight of the reviewer user is performed only when it is determined that the received evaluations satisfy the author reputation calculation threshold.

12. The method of claim 1 including, before the automatic generating of the aggregate assessment of the content of the review, determining whether the received evaluations satisfy a content rating threshold, and wherein the automatic generating of the aggregate assessment of the content of the review is performed only when it is determined that the received evaluations satisfy the content rating threshold.

13. The method of claim 12 wherein the content rating threshold is based at least in part on a weighted number of the evaluations received for the review from the evaluator users that is based on the reputation weights of the evaluator users in such a manner that an evaluation from a first evaluator user with a first reputation weight has a different impact on that weighted number of evaluations than an evaluation from a distinct second evaluator user with a distinct second reputation weight.

14. The method of claim 1 wherein each of the received evaluations includes quantitative assessments of the contents of the review for each of the multiple available content rating dimensions.

15. The method of claim 14 including, before the receiving of the evaluations of the review, determining the multiple available content rating dimensions.

16. The method of claim 1 including, before the receiving of the evaluations of the review, soliciting the evaluator users to provide evaluations of the review, the solicitations including indications of the multiple available content rating dimensions.

17. The method of claim 1 wherein the automatic generating of the aggregate assessment of the content of the review includes generating an aggregate assessment for each of the multiple available content rating dimensions.

18. The method of claim 17 including automatically generating an overall aggregate assessment of the review based at least in part on the multiple automatically generated aggregate assessments of the content of the review for the multiple available content rating dimensions.

19. The method of claim 1 wherein the automatically generated aggregate assessment of the content of the review is further based on a sales weighting for each of one or more of the evaluator users, the sales weighting of an evaluator user reflecting an amount of prior sales to that evaluator user.

20. The method of claim 1 including, after the automatic updating of the reputation weights for the evaluator users, ranking each evaluator user relative to other evaluator users based at least in part on automatically generated evaluator reputation scores of those evaluator users.

21. The method of claim 20 wherein the reputation weight for each of the evaluator users is based on a combination of quantity and quality of evaluations provided by that evaluator user, and including automatically generating a distinct evaluator reputation rating score for each of the evaluator users based solely on the quality of the evaluations provided by that evaluator user, and wherein the evaluator reputation scores used for the ranking are the evaluator reputation rating scores.

22. The method of claim 20 including providing visible feedback to users of the rankings of at least some of the evaluator users.

23. The method of claim 1 wherein at least some of the evaluator users each have multiple existing reputation weights that correspond to previous evaluations by those evaluator users of content of different categories, and including, before the automatic updating of the reputation weights for the evaluator users, determining a category of the review, and wherein the automatic updating of the reputation weights of evaluator users that have multiple existing reputation weights is performed for an existing reputation weight of that evaluator user for the determined category.

24. The method of claim 1 wherein at least some of the evaluator users each have multiple existing reputation weights that correspond to different types of activities previously performed by those evaluator users, and wherein the automatic updating of the reputation weights of evaluator users that have multiple existing reputation weights is performed for an existing reputation weight of that evaluator user corresponding to prior review evaluation activities of that evaluator user.

25. The method of claim 1 including, after the automatic updating of the reputation weights for the evaluator users, providing indications of the reputation weights for one or more of those evaluator users to one or more third-party computing devices to enable interactions by the third-party computing devices with those evaluator users based on those reputation weights.

26. The method of claim 25 wherein the one or more third-party computing devices are provided by an entity that is distinct from the Web merchant.

27. The method of claim 1 including receiving from one or more third-party computing devices information related to the reputations of one or more of the evaluator users, the received information based on interactions of those evaluator users with those third-party computing devices, and automatically updating the reputation weights for each of those evaluator users based on the received information.

28. The method of claim 1 wherein the review is information obtained from a blog authored by the reviewer user.

29. The method of claim 1 wherein the automatic generating of the aggregate assessment of the content of the review based on the existing reputation weights of the evaluator users is performed in a manner independent of the multiple additional users.

30. The method of claim 14 wherein the multiple available content rating dimensions include at least two of usefulness, accuracy, informativeness, and humorousness.

31. The method of claim 17 wherein the multiple available content rating dimensions include at least two of usefulness, accuracy, informativeness, and humorousness.

52. The method of claim 29 wherein the automatic updating of the existing reputation weights of the one or more evaluator users is performed in a manner independent of the multiple additional users.

32. A computer-readable medium whose contents cause a computing device to select information to provide to users based on reputations of evaluators of the information, by performing a method comprising:

receiving from a reviewer user a review related to an available item;

receiving evaluations of the review from each of multiple evaluator users, each received evaluation including a quantitative assessment of contents of the review for each of one or more of multiple content rating dimensions available for use in assessing the review, each of the evaluator users having an existing reputation weight based at least in part on previous evaluations;

automatically generating at least one aggregate assessment of the content of the review based at least in part on combining quantitative assessments from the received evaluations for the review, at least one of the generated aggregate assessments being further based on the reputation weights of the evaluator users in such a manner that a first quantitative assessment from a first evaluator user with a first reputation weight has a different impact on that generated aggregate assessment than that first quantitative assessment from a distinct second evaluator user with a distinct second reputation weight;

automatically updating the reputation weights for each of one or more of the evaluator users based on a relationship of the quantitative assessments from the evaluation of that evaluator user to the quantitative assessments from the evaluations of other of the evaluator users; and

determining whether to provide the review to another user based at least in part on one or more of the automatically generated aggregate assessments for the content of the review.

33. The computer-readable medium of claim 32 wherein the relationship of the quantitative assessments from the evaluation of an evaluator user to the quantitative assessments from the evaluations of other of the evaluator users that is used when automatically updating the reputation weight for that evaluator user is based on a degree of agreement between the quantitative assessments from the evaluation of the evaluator user and quantitative assessments from a consensus evaluation for the received evaluations.

34. The computer-readable medium of claim 32 wherein the reputation weights of the evaluator users that are used in the automatic generating of the at least one aggregate assessments of the content of the review are automatically generated based on the previous evaluations by those evaluator users.

35. The computer-readable medium of claim 32 wherein the automatic generating of the at least one aggregate assessments of the content of the review is further based in part on an existing reputation weight of the reviewer user from which the review was received.

36. The computer-readable medium of claim 35 wherein the existing reputation weight of the reviewer user is based on a degree of consistency between one or more of the automatically generated aggregate assessments of the content of the review and automatically generated aggregate assessments of the content of previous reviews received from the reviewer user, and wherein the method further comprises automatically updating the reputation weight of the reviewer user from which the review was received based at least in part on one or more of the automatically generated at least one aggregate assessments of the content of the review.

37. The computer-readable medium of claim 32 wherein the method further comprises, before the automatic generating of the at least one aggregate assessments of the content of the review, determining whether the received evaluations satisfy a content rating threshold, and wherein the automatic generating of the at least one aggregate assessments of the content of the review is performed only when it is determined that the received evaluations satisfy the content rating threshold.

38. The computer-readable medium of claim 32 wherein each of the received evaluations include quantitative assessments of the contents of the review for each of the multiple available content rating dimensions.

39. The computer-readable medium of claim 32 wherein the automatic generating of the at least one aggregate assessments of the content of the review includes generating multiple aggregate assessments that are each for one of the multiple available content rating dimensions, and wherein the method further comprises automatically generating an overall aggregate assessment of the review based at least in part on the multiple automatically generated aggregate assessments of the content of the review.

40. The computer-readable medium of claim 32 wherein the computer-readable medium is at least one of a memory of a computing device and a data transmission medium transmitting a generated data signal containing the contents.

41. The computer-readable medium of claim 32 wherein the contents are instructions that when executed cause the computing device to perform the method.

42. A computing device for selecting information to provide to users based on reputations of evaluators of the information, comprising:

one or more processors;

a content rater component configured to, when executed by at least one of the one or more processors;

receive from a reviewer user a review related to an item available from a Web merchant;

receive evaluations of the review from each of multiple evaluator users, each received evaluation including a quantitative assessment of contents of the review for each of one or more of multiple content rating dimensions available for use in assessing the review, each of the evaluator users having a single existing reputation weight for the Web merchant based at least in part on previous evaluations supplied by that evaluator user for multiple other reviews for items available from the Web merchant; and

automatically generate at least one aggregate assessment of the content of the review based at least in part on combining quantitative assessments from the received evaluations for the review, one or more of the generated aggregate assessments being further based on the single existing reputation weights of the evaluator users in such a manner that a first quantitative assessment from a first evaluator user with a first reputation weight has a different impact on that generated aggregate assessment than that first quantitative assessment from a distinct second evaluator user with a distinct second reputation weight;

an evaluator reputation assessor component configured to automatically update the single existing reputation weights for each of one or more of the evaluator users for the Web merchant based on a relationship of the quantitative assessments from the evaluation of that evaluator user to the quantitative assessments from the evaluations of other of the evaluator users; and

a content manager system configured to, when executed by at least one of the one or more processors, determine whether to provide the review to another user based at least in part on one or more of the automatically generated aggregate assessments for the content of the review.

43. The computing device of claim 42 wherein the relationship of the quantitative assessments from the evaluation of an evaluator user to the quantitative assessments from the evaluations of other of the evaluator users that is used when automatically updating the reputation weight for that evaluator user is based on a degree of agreement between the quantitative assessments from the evaluation of the evaluator user and quantitative assessments from a consensus evaluation for the received evaluations.

44. The computing device of claim 42 wherein the reputation weights of the evaluator users that are used in the automatic generating of the at least one aggregate assessments of the content of the review are automatically generated based on the previous evaluations by those evaluator users.

45. The computing device of claim 42 wherein the automatic generating of the at least one aggregate assessments of the content of the review is further based in part on an existing reputation weight of the reviewer user from which the review was received.

46. The computing device of claim 45 wherein the existing reputation weight of the reviewer user is based on a degree of consistency between one or more of the automatically generated aggregate assessments of the content of the review and automatically generated aggregate assessments of the content of previous reviews received from the reviewer user, and wherein the evaluator reputation assessor component is further configured to automatically update the reputation weight of the reviewer user from which the review was received based at least in part on one or more of the automatically generated at least one aggregate assessments of the content of the review.

47. The computing device of claim 42 wherein the content rater component is further configured to, before the automatic generating of the at least one aggregate assessments of the content of the review, determine whether the received evaluations satisfy a content rating threshold, and wherein the automatic generating of the at least one aggregate assessments of the content of the review is performed only when it is determined that the received evaluations satisfy the content rating threshold.

48. The computing device of claim 42 wherein each of the received evaluations include quantitative assessments of the contents of the review for each of the multiple available content rating dimensions.

49. The computing device of claim 42 wherein the automatic generating of the at least one aggregate assessments of the content of the review includes generating multiple aggregate assessments that are each for one of the multiple available content rating dimensions, and wherein the content rater component is further configured to automatically generate an overall aggregate assessment of the review based at least in part on the multiple automatically generated aggregate assessments of the content of the review.

50. The computing device of claim 42 wherein the content rater component and the evaluator reputation assessor component each include software instructions for execution in memory of the computing device.

51. The computing device of claim 42 wherein the content rater component consists of means for receiving from a reviewer user a review related to an available item, for receiving evaluations of the review from each of multiple evaluator users, each received evaluation including a quantitative assessment of contents of the review for each of one or more of multiple content rating dimensions available for use in assessing the review, each of the evaluator users having an existing reputation weight based at least in part on previous evaluations, and for automatically generating at least one aggregate assessment of the content of the review based at least in part on combining quantitative assessments from the received evaluations for the review, at least one of the generated aggregate assessments being further based on the reputation weights of the evaluator users in such a manner that a first quantitative assessment from a first evaluator user with a first reputation weight has a different impact on that generated aggregate assessment than that first quantitative assessment from a distinct second evaluator user with a distinct second reputation weight, wherein the evaluator reputation assessor component consists of means for automatically updating the reputation weights for each of one or more of the evaluator users based on a relationship of the quantitative assessments from the evaluation of that evaluator user to the quantitative assessments from the evaluations of other of the evaluator users, and wherein the content manager system consists of means for determining whether to provide the review to another user based at least in part on one or more of the automatically generated aggregate assessments for the content of the review.