Methods, systems, and media for applying scores and ratings to web pages, web sites, and content for safe and effective online advertising

• US 8,732,017 B2
• Filed: 06/01/2011
• Issued: 05/20/2014
• Est. Priority Date: 06/01/2010
• Status: Active Grant

• Current Assignee: Integral Ad Science Incorporated (Integral Ad Science Holding Corp.)
• Original Assignee: Integral Ad Science Incorporated (Integral Ad Science Holding Corp.)
• Inventors: Attenberg, Joshua M., Provost, Foster J.
• Primary Examiner(s): Gartland, Scott D
• Application Number:
  US13/151,146
• Publication Number:
  US 20120010927A1
• Time in Patent Office: 1,084 Days
• Field of Search: 705/14.6, 705/14.73
• US Class Current: 705/14.6
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0263 based upon Internet or website rating
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06Q 30/0282 Rating or review of business operators or products

Abstract

Systems, methods, and media for rating websites for safe advertising are provided. In accordance with some embodiments of the disclosed subject matter, the method comprises: extracting one or more features from a piece of web content; applying a plurality of statistical models to the extracted features to generate a plurality of ordinomial estimates, wherein each ordinomial estimate represents a probability that the web content is a member of one of a plurality of severity groups; determining a posterior ordinomial estimate for the web content by combining the plurality of ordinomial estimates; generating a risk rating that encodes severity and confidence based on the determined posterior ordinomial estimate, wherein the risk rating identifies whether the web content is likely to contain objectionable content of a given category; and providing the risk rating for determining whether an advertisement should be associated with the web content.

Claims

1. A method for rating webpages for safe advertising, the method comprising:

extracting one or more features from a piece of web content;

applying, using a hardware processor, a plurality of statistical models to the extracted features to generate a plurality of ordinomial estimates, wherein each ordinomial estimate represents a probability that the web content is a member of one of a plurality of severity groups;

determining a posterior ordinomial estimate for the web content by combining the plurality of ordinomial estimates;

generating a risk rating that encodes severity and confidence based on the determined posterior ordinomial estimate, wherein the risk rating identifies whether the web content is likely to contain objectionable content of a given category; and

providing the risk rating for determining whether an advertisement should be associated with the web content.

2. The method of claim 1, further comprising:

determining a plurality of posterior ordinomial estimates at a plurality of times for the web content; and

determining an expected posterior ordinomial estimate by combining the plurality of posterior ordinomial estimates over the plurality of times.

3. The method of claim 2, further comprising:

extracting a uniform resource locator from the one or more features;

assembling a first set of posterior ordinomial estimates from the plurality of posterior ordinomial estimates based on the uniform resource locator; and

determining the expected posterior ordinomial estimate by combining the first set of posterior ordinomial estimates over the plurality of times.

4. The method of claim 2, further comprising determining that the web content belongs to a sitelet, wherein the sitelet includes a plurality of web pages.

5. The method of claim 4, further comprising:

determining a sitelet ordinomial by aggregating the plurality of posterior ordinomial estimates associated with each of the plurality of web pages; and

generating a sitelet rating based on the aggregated plurality of posterior ordinomials.

6. The method of claim 5, further comprising:

comparing the sitelet ordinomial with the plurality of posterior ordinomial estimates associated with each of the plurality of web pages belonging to the sitelet; and

determining whether to store at least one of the sitelet ordinomial and the plurality of posterior ordinomial estimates based on the comparison and a sensitivity value.

7. The method of claim 1, further comprising:

collecting a plurality of ratings associated with a plurality of pieces of web content, wherein the plurality of ratings includes ordinal ratings and real-valued ratings; and

determining an aggregate rating for the plurality of pieces of web content based on the collected plurality of ratings.

8. The method of claim 7, further comprising normalizing the aggregate rating by mapping the aggregate rating to an index-scaled rating.

9. The method of claim 8, further comprising:

applying a severity weight to the index-scaled rating; and

generating a severity-weighted index-scaled rating for the plurality of pieces of web content.

10. The method of claim 1, further comprising generating a combined risk rating by combining the generated risk rating that encodes whether the web content is likely to contain objectionable content of the given category with a second risk rating that encodes whether the web content is likely to contain objectionable content of a second category.

11. A system for rating webpages for safe advertising, the system comprising:

a processor that;

extracts one or more features from a piece of web content;

applies a plurality of statistical models to the extracted features to generate a plurality of ordinomial estimates, wherein each ordinomial estimate represents a probability that the web content is a member of one of a plurality of severity groups;

determines a posterior ordinomial estimate for the web content by combining the plurality of ordinomial estimates;

generates a risk rating that encodes severity and confidence based on the determined posterior ordinomial estimate, wherein the risk rating identifies whether the web content is likely to contain objectionable content of a given category; and

provides the risk rating for determining whether an advertisement should be associated with the web content.

12. The system of claim 11, wherein the processor is further configured to:

determine a plurality of posterior ordinomial estimates at a plurality of times for the web content; and

determine an expected posterior ordinomial estimate by combining the plurality of posterior ordinomial estimates over the plurality of times.

13. The system of claim 12, wherein the processor is further configured to:

extract a uniform resource locator from the one or more features;

assemble a first set of posterior ordinomial estimates from the plurality of posterior ordinomial estimates based on the uniform resource locator; and

determine the expected posterior ordinomial estimate by combining the first set of posterior ordinomial estimates over the plurality of times.

14. The system of claim 12, wherein the processor is further configured to determine that the web content belongs to a sitelet, wherein the sitelet includes a plurality of web pages.

15. The system of claim 14, wherein the processor is further configured to:

determine a sitelet ordinomial by aggregating the plurality of posterior ordinomial estimates associated with each of the plurality of web pages; and

generate a sitelet rating based on the aggregated plurality of posterior ordinomials.

16. The system of claim 15, wherein the processor is further configured to:

compare the sitelet ordinomial with the plurality of posterior ordinomial estimates associated with each of the plurality of web pages belonging to the sitelet; and

determine whether to store at least one of the sitelet ordinomial and the plurality of posterior ordinomial estimates based on the comparison and a sensitivity value.

17. The system of claim 11, wherein the processor is further configured to:

collect a plurality of ratings associated with a plurality of pieces of web content, wherein the plurality of ratings includes ordinal ratings and real-valued ratings; and

determine an aggregate rating for the plurality of pieces of web content based on the collected plurality of ratings.

18. The system of claim 17, wherein the processor is further configured to normalize the aggregate rating by mapping the aggregate rating to an index-scaled rating.

19. The system of claim 18, wherein the processor is further configured to:

apply a severity weight to the index-scaled rating; and

generate a severity-weighted index-scaled rating for the plurality of pieces of web content.

20. The system of claim 11, wherein the processor is further configured to generate a combined risk rating by combining the generated risk rating that encodes whether the web content is likely to contain objectionable content of the given category with a second risk rating that encodes whether the web content is likely to contain objectionable content of a second category.

21. A non-transitory computer-readable medium containing computer-executable instructions that, when executed by a processor, cause the processor to perform a method for rating webpages for safe advertising, the method comprising:

extracting one or more features from a piece of web content;

applying a plurality of statistical models to the extracted features to generate a plurality of ordinomial estimates, wherein each ordinomial estimate represents a probability that the web content is a member of one of a plurality of severity groups;

determining a posterior ordinomial estimate for the web content by combining the plurality of ordinomial estimates;

generating a risk rating that encodes severity and confidence based on the determined posterior ordinomial estimate, wherein the risk rating identifies whether the web content is likely to contain objectionable content of a given category; and

providing the risk rating for determining whether an advertisement should be associated with the web content.