System and method for rating electronic documents

• US 20050223002A1
• Filed: 03/30/2004
• Published: 10/06/2005
• Est. Priority Date: 03/30/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Agarwal, Sumit, Rajaram, Gokul, Wiseman, Leora Ruth
• Granted Patent: US 7,533,090 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0258 Registration
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A system and method for rating an electronic document such as an advertisement. Rating information is received from one or more evaluators. A signal relevant to a criteria is received and a determination is made whether to deliver the document in response to the signal based on the criteria and the rating information from the one or more evaluators.

Claims

1. A method comprising the steps of:

receiving rating information associated with a document from one or more evaluators;

receiving a signal relevant to a criteria; and

determining whether to deliver the document in response to the signal based on the criteria and the rating information from the one or more evaluators.

2. The method of claim 1, further comprising:

selecting a plurality of evaluators to rate the document; and

passing the document to the plurality of evaluators for rating.

3. The method of claim 1, wherein the signal is a request received from an entity, the entity is associated with a suitability standard, and the determining action comprises determining whether the document satisfies the suitability standard.

4. The method of claim 1, further comprising:

determining an aggregate rating based on the rating information, wherein the action of determining whether to deliver the document is further based on the aggregate rating of the document.

5. The method of claim 3, further comprising:

determining an aggregate rating based on the rating information; and

selecting the electronic document based on the aggregate rating and the suitability standard.

6. The method of claim 3, further comprising:

storing the suitability standard in a database; and

processing the database to determine the suitability standard.

7. The method of claim 1, further comprising:

identifying at least one trust score, wherein each trust score is associated with a specific one of a plurality of evaluators;

determining an aggregate rating based on the rating information and the at least one trust score, wherein determining whether to deliver the document is further based on the aggregate rating.

8. The method of claim 7, wherein at least one trust score is based on the geographical location of the associated specific one of the plurality of evaluators.

9. The method of claim 7, wherein at least one trust score is based on one or more prior content ratings received from the associated specific one of the plurality of evaluators.

10. The method of claim 7, wherein at least one trust score is based on an industry associated with the associated specific one of the plurality of evaluators.

11. The method of claim 7, wherein at least one trust score is based on a rating deviation of an evaluator, wherein the rating deviation is based on a comparison of (i) rating information for one or more documents received from the evaluator and (ii) rating information for the one or more documents received from one or more other evaluators.

12. The method of claim 7, wherein at least one trust score is based on a rating deviation of an evaluator, wherein the rating deviation is based on a comparison of (i) rating information for one or more documents received from the evaluator and (ii) aggregate ratings for the one or more documents.

13. The method of claim 7, wherein the trust score of an evaluator is based on rating information previously received from the evaluator for one or more documents.

14. The method of claim 7, further comprising:

determining one or more revised trust scores for one or more of the plurality of evaluators; and

determining a revised aggregate rating based on the one or more revised trust scores.

15. The method of claim 7, wherein at least one of the plurality of trust scores is based on at least one of the geographical location of the associated evaluator and prior rating information received from the evaluator.

16. The method of claim 1, wherein the document is an advertisement.

17. The method of claim 1, wherein the document comprises at least one of a web page, content that can be used in a web page, and a program.

18. The method of claim 1, further comprising the action of selecting a plurality of evaluators to evaluate the document.

19. The method of claim 18, wherein the plurality of evaluators are selected using a random selection algorithm.

20. The method of claim 1, wherein the criteria comprises a sensitivity score.

21. The method of claim 1, further comprising

receiving new rating information for the document;

and

processing the new rating information to determine a revised rating associated with the document.

22. The method of claim 1, wherein the aggregate rating comprises one or more subject ratings, each associated with an evaluation criterion.

23. The method of claim 22, wherein the evaluation criterion comprises at least one of sexual content, violent content, adult content, and targeted age.

24. The method of claim 1, wherein the aggregate rating comprises a quantity.

25. The method of claim 1, wherein the aggregate rating comprises a mean of the rating information.

26. The method of claim 1, wherein the aggregate rating comprises a mode of the rating information.

27. The method of claim 1, wherein the aggregate rating comprises a median of the rating information.

28. The method of claim 1, wherein the request is received by at least one of a content provider and a user.

29. A computer-readable medium encoded with computer program code to rate an electronic document, the program code effective to perform the following:

receive rating information associated with a document from one or more evaluators;

receive a signal relevant to a criteria; and

determine whether to deliver the document in response to the signal based on the criteria and the rating information from the one or more evaluators.

30. A system comprising:

rating receiving means for receiving rating information associated with a document from one or more evaluators;

signal receiving means for receiving a signal relevant to a criteria; and

determination means for determining whether to deliver the document in response to the signal based on the criteria and the rating information from the one or more evaluators.

31. A system comprising:

a rating receiving device that receives rating information associated with a document from one or more evaluators;

a signal receiving device that receives a signal relevant to a criteria; and

a determination device that determines whether to deliver the document in response to the signal based on the criteria and the rating information from the one or more evaluators.

32. A method comprising the steps of:

receiving rating information for an electronic document from a plurality of evaluators, wherein at least one evaluator is associated with a trust score; and

determining an aggregate content rating for the electronic document by aggregating the rating information from at least two of the plurality of evaluators, wherein the rating information for at least one evaluator is considered in association with the trust score for that evaluator.

33. A method comprising the steps of:

transmitting a signal to a server, the server interpreting the signal as being associated with a criteria for a document and determining whether a document is to be delivered based in part on rating information received from a plurality of evaluators for one or more documents and whether the one or more documents relate to the criteria; and

receiving a document in response to the signal.

34. A method comprising:

receiving one or more documents from a server system; and

providing rating information to the server system for use by the server system in determining whether to deliver the one or more documents in response to requests for the one or more documents based on the rating provided.

35. A method comprising the steps of:

receiving content rating information associated with an advertisement from a plurality of evaluators, wherein at least one of the plurality of advertisers is a partner advertisement listings provider;

determining an aggregate rating for the advertisement based on the rating information;

receiving a request for electronic delivery of one or more advertisements that satisfy ad content criteria; and

determining whether to deliver the first advertisement in response to the request based on the ad content criteria and the aggregate rating.