System and method for rating electronic documents

• US 7,533,090 B2
• Filed: 03/30/2004
• Issued: 05/12/2009
• Est. Priority Date: 03/30/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Agarwal, Sumit, Rajaram, Gokul, Wiseman, Leora Ruth
• Primary Examiner(s): Cottingham; John R.
• Assistant Examiner(s): Pham; Michael
• Application Number:
  US10/812,417
• Publication Number:
  US 20050223002A1
• Time in Patent Office: 1,869 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0258 Registration
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A system and method for rating an electronic document such as an advertisement. Rating information is received from one or more evaluators. A signal relevant to a criteria is received and a determination is made whether to deliver the document in response to the signal based on the criteria and the rating information from the one or more evaluators.

Claims

1. A method comprising the steps of:

receiving, at a server comprising a central processor, rating information associated with contents of a document from one or more evaluators;

identifying, at the server, a trust score for each of the one or more evaluators, each trust score for each of the one or more evaluators being a measure of an extent to which a respective evaluator'"'"'s rating information is considered in determining an aggregate rating, each trust score for each of the one or more evaluators based on factors associated with the respective evaluator,

wherein each trust score for each of the one or more evaluators is based on a rating deviation of the respective evaluator, wherein the rating deviation is based on a comparison of (i) rating information for one or more documents received from the respective evaluator and (ii) rating information for the one or more documents received from one or more other evaluators;

identifying, at the server, a sensitivity score, the sensitivity score indicating a level of conservatism in determining the aggregate rating, wherein a first level sensitivity indicates a higher degree of conservatism in determining the aggregate rating;

determining, at the server, the aggregate rating for the document based on the rating information associated with contents of the document from the one or more evaluators, each trust score for each of the one or more evaluators, and the sensitivity score;

receiving a request from a recipient;

identifying, at the server, a suitability profile associated with the recipient, where the suitability profile determines whether any information relating to preferences of the recipient are stored in a suitability database, and is used to select the document appropriate for the recipient; and

determining whether to deliver the document in response to the request for the document based on the suitability profile of the recipient, and the aggregate rating of the document,

wherein determining whether to deliver the document further comprises determining whether to deliver the document based on whether the aggregate rating satisfies the suitability profile of the recipient; and

delivering the document to the recipient if the aggregate rating satisfies the suitability profile of the recipient.

2. The method of claim 1, wherein each trust score for each of the one or more evaluators is based on a geographical location of the respective evaluator.

3. The method of claim 1, wherein each trust score for each of the one or more evaluators is based on one or more prior content ratings received from the respective evaluator.

4. The method of claim 1, wherein each trust score for each of the one or more evaluators is based on an industry associated with the respective evaluator.

5. The method of claim 1, wherein each trust score for each of the one or more evaluators is based on a rating deviation of the respective evaluator, wherein the rating deviation is based on a comparison of (i) rating information for one or more documents received from the evaluator and (ii) aggregate ratings for the one or more documents.

6. The method of claim 1, wherein each trust score for each of the one or more evaluators is based on rating information previously received from the respective evaluator for one or more documents.

7. The method of claim 1, further comprising:

determining one or more revised trust scores for one or more of the one or more evaluators; and

determining a revised aggregate rating based on the one or more revised trust scores.

8. The method of claim 1, wherein the document is an advertisement.

9. The method of claim 1, wherein the document comprises at least one of a web page, content that can be used in a web page, or a program.

10. The method of claim 1, wherein the one or more evaluators are selected using a random selection algorithm.

11. The method of claim 1, further comprising:

receiving new rating information for the document; and

processing the new rating information to determine a revised rating associated with the document.

12. The method of claim 1, wherein the aggregate rating comprises one or more subject ratings, each associated with an evaluation criterion.

13. The method of claim 12, wherein the evaluation criterion comprises at least one of sexual content, violent content, adult content, or targeted age.

14. The method of claim 1, wherein the aggregate rating comprises a quantity.

15. The method of claim 1, wherein the aggregate rating comprises a mean of the rating information.

16. The method of claim 1, wherein the aggregate rating comprises a mode of the rating information.

17. The method of claim 1, wherein the aggregate rating comprises a median of the rating information.

18. A system comprising:

a server including a central processor;

means for receiving, at the server, rating information associated with contents of a document from one or more evaluators;

means for identifying, at the server, a trust score for each of the one or more evaluators, each trust score for each of the one or more evaluators being a measure of an extent to which a respective evaluator'"'"'s rating information is considered in determining an aggregate rating, each trust score for each of the one or more evaluators based on factors associated with the respective evaluator,

wherein each trust score for each of the one or more evaluators is based on a rating deviation of the respective evaluator, wherein the rating deviation is based on a comparison of (i) rating information for one or more documents received from the respective evaluator and (ii) rating information for the one or more documents received from one or more other evaluators;

means for identifying, at the server, a sensitivity score, the sensitivity score indicating a level of conservatism in determining the aggregate rating, wherein a first level sensitivity indicates a higher degree of conservatism in determining the aggregate rating;

means for determining, at the server, the aggregate rating for the document based on the rating information associated with contents of the document, each trust score, for each of the one or more evaluators and the sensitivity score;

means for receiving a request from a recipient;

means for identifying, at the server, a suitability profile associated with the recipient, where the suitability profile determines whether any information relating to preferences of the recipient are stored in a suitability database, and is used to select the document appropriate for the recipient; and

means for determining whether to deliver the document in response to the request for the document based on the suitability profile of the recipient, and the aggregate rating of the document,

wherein the means for determining whether to deliver the document further comprises determining whether to deliver the document based on whether the aggregate rating satisfies the suitability profile of the recipient; and

delivering the document to the recipient if the aggregate rating satisfies the suitability profile of the recipient.