Auditing a website with page scanning and rendering techniques

• US 8,365,062 B2
• Filed: 10/25/2010
• Issued: 01/29/2013
• Est. Priority Date: 11/02/2008
• Status: Active Grant

• Current Assignee: ObservePoint, Inc.
• Original Assignee: ObservePoint, Inc.
• Inventors: Seolas, Robert K., Pestana, John, Broadbent, J. Tyler, Zinn, Richard
• Primary Examiner(s): Hutton, Jr., Doug
• Assistant Examiner(s): DYER, ANDREW R
• Application Number:
  US12/911,626
• Publication Number:
  US 20110041090A1
• Time in Patent Office: 827 Days
• Field of Search: 715/206, 715/234, 715/276, 715/277, 715/733, 715/734, 715/738, 715/760
• US Class Current: 715/206
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

An automated website analysis system includes mechanisms for automatically reviewing a website and identifying various features of the website. In one implementation, the analysis system can rely at least in part on the actual downloading and rendering of each page in the website to determine not only aspects of website content and layout, but also whether various content, such as third party objects or source code, execute as intended. Additional features include recording user input that may be required to progress past web pages where automated site mapping would ordinarily stall. The analysis system can identify various items of interest about each web page to the website owner, including web page parent/child relationships, as well as the extent to which the given page content is consistent with its metadata descriptions.

Claims

1. In a computerized environment comprising a rendering system and a website having one or more web pages a method of analyzing the website using both page scan and page rendering techniques, comprising:

requesting, from a server, one or more web pages of an identified website;

receiving a response from the server, the response including page code that defines the one or more web pages of the identified website;

scanning the page code of the one or more web pages without rendering the one or more web pages during scanning to identify scanned web page information;

rendering the one or more web pages, wherein the results of the rendered one or more web pages comprise-rendered web page information;

analyzing the one or more web pages by comparing the scanned web page information and the rendered web page information; and

displaying a report for the website, wherein the report;

is based at least in part on the comparison of the scanned web page information and the rendered web page information,

includes a combination of scanned web page information and rendered web page information for the one or more web pages,

includes a reference to a plurality of links or analytic objects present in the one or more web pages, and

includes an indication that one or more links or analytics objects of the plurality of links or analytic objects included in the page code, as indicated by the scanned web page information, was not loaded or executed during the rendering of the one or more web pages, as indicated by the rendered web page information.

2. The method as recited in claim 1, wherein the report further comprises:

a comparison between one or more of objects expected to be rendered as indicated by the scanned web page information and objects actually rendered as indicated by the rendered web page information.

3. The method as recited in claim 2, wherein the report further comprises an indication whether at least a portion of the page code was interpreted during rendering as expected based on a comparison of the rendered web page information and the scanned web page information.

4. The method as recited in claim 1, further comprising generating the report for the one or more web pages based on a comparison of expected information about the one or more web pages with at least one of:

the scanned web page information;

or

the rendered web page information.

5. The method as recited in claim 4, wherein the report indicates whether content of the one or more web pages is consistent with HTML meta tags included within the content of the one or more web pages.

6. The method as recited in claim 4, further comprising:

wherein the expected information comprises a privacy policy that defines the type of information that an owner of the website states it will and will not request; and

determining whether the one or more web pages requests user information that is consistent with the privacy policy.

7. The method as recited in claim 4, further comprising:

determining from the comparison of expected information whether one or more key words used for advertising by an agent of the website were identified in the scanned web page;

wherein the report reflects the determination for key words.

8. The method as recited in claim 4, further comprising:

determining, based on the rendered web page information, whether one or more third party objects in the one or more web pages were executed in accordance with the expected information;

wherein the report reflects the determination for third party objects.

9. The method as recited in claim 4, further comprising:

determining, based on the rendered page information, whether one or more expected values were passed during the execution of one or more executable routines found in the web page code;

wherein the report reflects the determination of any passed values.

10. In a computerized environment, non-transitory computer computer-readable storage device medium having computer executable instructions stored thereon that, when executed cause one or more processors in a computer system to perform a method of automatically analyzing a website health using page rendering techniques, comprising:

requesting, from a server, one or more web pages of an identified website;

receiving a response from the server, the response including page code that defines the one or more web pages of the identified website;

scanning the page code of the one or more web pages without rendering the one or more web pages during scanning to identify scanned web page information;

rendering the one or more web pages, wherein the results of the rendered one or more web pages comprise rendered web page information;

analyzing the one or more web pages by comparing the scanned web page information and the rendered web page information; and

displaying a report for the website, wherein the report;

is based at least in part on the comparison of the scanned web page information and the rendered web page information,

includes a combination of scanned web page information and rendered web page information for the one or more web pages,

includes a reference to a plurality of links or analytic objects present in the one or more web pages, and

includes an indication that one or more links or analytics objects of the plurality of links or analytic objects included in the page code, as indicated by the scanned web page information, was not loaded or executed during the rendering of the one or more web pages, as indicated by the rendered web page information.

11. In a computerized environment comprising a rendering system and a website having one or more web pages that have one or more parent and child relationships, a method of the rendering system automatically generating a site map that shows one or more of the parent and child relationships using page rendering techniques, comprising:

receiving a request to generate a site map, wherein the request comprises one or more end user provided processing parameters, and an identified website;

requesting one or more web pages of the identified website;

receiving the one or more web pages, each web page comprising page code that defines the web page;

processing the one or more web pages corresponding to the identified website in accordance with the user provided processing parameters including;

scanning the page code of one or more web pages without rendering the one or more web pages during scanning to identify scanned web page information;

rendering the one or more web pages, wherein the results of the rendered one or more web pages comprise rendered web page information;

analyzing the one or more web pages by comparing the scanned web page information and the rendered web page information; and

generating a site map for display in a user interface, wherein the site map;

displays the scanned web page information including one or more link relationships between a plurality of web pages in the website,

displays at least a portion of the rendered web page information including rendering results for the rendered one or more web pages, and

includes a reference to a plurality of links present in the one or more web pages, and

includes an indication that one or more links of the plurality of links included in the page code, as indicated by the scanned web page information, was not loaded during the rendering of the one or more web pages, as indicated by the rendered web page information.

12. The method as recited in claim 11, further comprising:

identifying at least one parent web page in the identified website that requires user input to process the at least one parent web page; and

retrieving pre-recorded user input for the at least one parent web page, wherein the pre-recorded user input allows the at least one parent web page to be processed.

13. The method as recited in claim 12, further comprising:

providing one or more interfaces that allow an end user to provide input for the at least one parent web page; and

storing the user'"'"'s input as the pre-recorded user input.

14. The method as recited in claim 11, further comprising, after completing processing of the at least one parent web page, automatically identifying one or more child web pages that are linked to the at least one parent web page.

15. The method as recited in claim 11, wherein the processing parameters comprise one or more of:

a level of depth for creating the site map;

or

a maximum number of web pages of the website to process when creating the site map.

16. The method as recited in claim 11, further comprising:

wherein the processing parameters comprise a set of one or more link filters; and

excluding one or more identified links from the site map based on the link filter.

17. The method as recited in claim 11, wherein the processing parameters comprise one or more of:

a speed value that limits the number of requests made by a rendering system to the web server over a time interval;

or

a frequency for scanning the website.

18. The method as recited in claim 11, wherein the processing parameters comprise an indication to prevent logging requests for tracking pixels while rendering a web page.

19. The method as recited in claim 11, wherein the processing parameters comprise a selection for checking the spelling of text of a web page.

20. The method as recited in claim 11, wherein the site map further displays:

a visible rendering of each of the one or more web pages and a speed at which each web page was rendered.