Automatic identification of unreliable user ratings
First Claim
1. A method for a computing system to automatically detect unreliability of a rating from a user based at least in part on a pattern identified using prior ratings of the user, the method comprising:
- receiving an indication of a rating that reflects an evaluation by a user related to a target item that is available for acquisition, the indicated rating being a rating by the user that is of a review of the target item provided by another user;
obtaining one or more indications of multiple other ratings by the user that reflect evaluations related to other items;
automatically analyzing by the computing system the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating, wherein the computing system is configured with instructions to perform the automatic analyzing of the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating; and
if the unreliability of the indicated rating is identified based on the one or more patterns, providing an indication that the indicated rating is unreliable so as to exclude use of the indicated rating as part of an aggregate rating related to the target item, and otherwise including the use of the indicated rating as part of the aggregate rating related to the target item.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques are described for enhancing the reliability of information received from users in various ways, such as by analyzing votes or other ratings supplied by a user in order to identify fraudulent and other unreliable ratings. Such analysis may be based at least in part on prior ratings submitted by the user, such as based on one or more patterns indicating that the user has a bias for or against one or more of various aspects. Unreliable ratings can then be excluded from use in various ways. This abstract is provided to comply with the rules requiring it, and is submitted with the intention that it not limit the scope of the claims.
-
Citations
102 Claims
-
1. A method for a computing system to automatically detect unreliability of a rating from a user based at least in part on a pattern identified using prior ratings of the user, the method comprising:
-
receiving an indication of a rating that reflects an evaluation by a user related to a target item that is available for acquisition, the indicated rating being a rating by the user that is of a review of the target item provided by another user; obtaining one or more indications of multiple other ratings by the user that reflect evaluations related to other items; automatically analyzing by the computing system the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating, wherein the computing system is configured with instructions to perform the automatic analyzing of the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating; and if the unreliability of the indicated rating is identified based on the one or more patterns, providing an indication that the indicated rating is unreliable so as to exclude use of the indicated rating as part of an aggregate rating related to the target item, and otherwise including the use of the indicated rating as part of the aggregate rating related to the target item. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35)
-
-
36. A method for a computing system to automatically detect unreliability of a rating from a user based at least in part on a pattern identified using prior ratings of the user, the method comprising:
-
receiving an indication of a rating that reflects an evaluation by a user related to a target item that is available for acquisition; obtaining one or more indications of multiple other ratings by the user that reflect evaluations related to other items; automatically analyzing by the computing system the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating, the one or more patterns including a pattern indicating a user is generating blocks of ratings, wherein the blocks of ratings by the user are indicated by one of the patterns as being prolific block rating by a new rater, and wherein the computing system is configured with instructions to perform the automatic analyzing of the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating, the one or more patterns including a pattern indicating a user is generating blocks of ratings; and if the unreliability of the indicated rating is identified based on the one or more patterns, providing an indication that the indicated rating is unreliable so as to exclude use of the indicated rating as part of an aggregate rating related to the target item, and otherwise including the use of the indicated rating as part of the aggregate rating related to the target item. - View Dependent Claims (37, 38, 39, 40, 41, 42, 43, 44)
-
-
45. A method for a computing system to automatically detect unreliability of a rating from a user based at least in part on a pattern identified using prior ratings of the user, the method comprising:
-
receiving an indication of a rating that reflects an evaluation by a user related to a target item that is available for acquisition, the indicated rating being a numerical rating of a usefulness of a review of the target item by another user; obtaining one or more indications of multiple other ratings by the user that reflect evaluations related to other items; automatically analyzing by the computing system the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating, wherein the computing system is configured with instructions to perform the automatic analyzing of the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating; and if the unreliability of the indicated rating is identified based on the one or more patterns, providing an indication that the indicated rating is unreliable so as to exclude use of the indicated rating as part of an aggregate rating related to the target item, and otherwise including the use of the indicated rating as part of the aggregate rating related to the target item. - View Dependent Claims (46, 47, 48, 49, 50, 51)
-
-
52. A method for a computing system to automatically detect unreliability of a rating from a user based at least in part on a pattern identified using prior ratings of the user, the method comprising:
-
receiving an indication of a rating that reflects an evaluation by a user related to a target item that is available for acquisition, and wherein the computing system is operated on behalf of an online retailer who offers the target item; obtaining one or more indications of multiple other ratings by the user that reflect evaluations related to other items; automatically analyzing by the computing system the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating, the automatic analyzing including automatically identifying the indicated rating as being unreliable if prior activities of the user do not satisfy specified criteria, wherein the specified criteria include a specified number of one or more prior purchases by the user from the online retailer, and wherein the computing system is configured with instructions to perform the automatic analyzing of the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating, the automatic analyzing including automatically identifying the indicated rating as being unreliable if prior activities of the user do not satisfy specified criteria; and if the unreliability of the indicated rating is identified based on the one or more patterns, providing an indication that the indicated rating is unreliable so as to exclude use of the indicated rating as part of an aggregate rating related to the target item, and otherwise including the use of an indicated rating as part of the aggregate rating related to the target item. - View Dependent Claims (53, 54, 55, 56, 57, 58, 59, 60)
-
-
61. A method for a computing system of an online retailer to automatically detect fraudulent votes received from customers who are evaluating item reviews supplied by other customers, the method comprising:
-
presenting to a first customer a first item review for an item available from the merchant, the first item review provided by a distinct second customer; receiving a first vote from the first customer that reflects an evaluation by the first customer of a usefulness of the first item review; automatically assessing by the computing system whether the first vote is fraudulent by, retrieving information regarding a vote history of the first customer that includes multiple prior votes of the first customer, each prior vote reflecting an evaluation by the first customer of a usefulness of an item review distinct from the first item review that was provided by a customer distinct from the first customer; retrieving information regarding multiple prior votes of other customers that reflect evaluations by those other customers of usefulness of the first item review; evaluating unreliability of the first vote by comparing the first vote and the vote history of the first customer to multiple predefined patterns associated with unreliable vote histories and by comparing the first vote to the information regarding the prior votes of the other customers for the first item review; and if the evaluated unreliability of the first vote exceeds a predefined unreliability threshold, identifying the first vote as fraudulent, and otherwise not identifying the first vote as fraudulent, wherein the computing system is configured with instructions to perform the automatic assessing by retrieving information regarding a vote history of the first customer that includes multiple prior votes of the first customer, each prior vote reflecting an evaluation by the first customer of a usefulness of an item review distinct from the first item review that was provided by a customer distinct from the first customer, retrieving information regarding multiple prior votes of other customers that reflect evaluations by those other customers of usefulness of the first item review, and evaluating unreliability of the first vote by comparing the first vote and the vote history of the first customer to multiple predefined patterns associated with unreliable vote histories and by comparing the first vote to the information regarding the prior votes of the other customers for the first item review; and when the first vote is identified as fraudulent, automatically excluding by the computing system the first vote from a group of votes used to rate the usefulness of the first item review, wherein the computing system is further configured with instructions to perform the automatic excluding of the first votes when the first vote is identified as fraudulent. - View Dependent Claims (62, 63, 64, 65, 66, 67, 68, 69, 70)
-
-
71. A computing system configured to automatically detect unreliability of a rating from a user based at least in part on a pattern identified using prior ratings of the user, comprising:
-
one or more memories; a reliability assessor system configured to include software instructions stored in at least one of the one or more memories and that when executed by a processor cause the computing system to receive an indication of a rating that reflects an evaluation by a user related to a target item that is available for acquisition, the indicated rating being a rating by the user that is of a review of the target item provided by another user, to obtain one or more indications of multiple other ratings by the user that reflect evaluations related to other items, and to automatically analyze the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating; and a content manager system configured to include software instructions stored in at least one of the one or more memories and that when executed by a processor cause the computing system to manage the indicated rating by, if the unreliability of the indicated rating is identified based on the one or more patterns, excluding use of the indicated rating as part of an aggregate rating related to the target item, and otherwise including the use of the indicated rating as part of the aggregate rating related to the target item. - View Dependent Claims (72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87)
-
-
88. A computer-readable storage medium whose contents are instructions that when executed cause a computing device to automatically detect unreliability of a rating from a user based at least in part on a pattern using prior ratings of the user, by performing a method comprising:
-
receiving an indication of a rating that reflects an evaluation by a user related to a target item that is available for acquisition, the indicated rating being a rating by the user that is of a review of the target item provided by another user; obtaining one or more indications of multiple other ratings by the user that reflect evaluations related to other items; automatically analyzing the indicated rating and the other ratings to determine whether one or more patterns in the indicated rating and other ratings identify an unreliability of the indicated rating; and if the unreliability of the indicated rating is identified based on the one or more patterns, indicating that the indicated rating is unreliable so as to exclude use of the indicated rating as part of an aggregate rating related to the target item, and otherwise including the use of the indicated rating as part of the aggregate rating related to the target item. - View Dependent Claims (89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102)
-
Specification