Method and system for automated analysis and transformation of web pages

• US 8,788,577 B2
• Filed: 05/31/2011
• Issued: 07/22/2014
• Est. Priority Date: 06/22/2010
• Status: Active Grant

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Akamai Technologies, Inc.
• Inventors: Podjarny, Guy
• Primary Examiner(s): Ibrahim, Mohamed
• Application Number:
  US13/149,025
• Publication Number:
  US 20110314091A1
• Time in Patent Office: 1,148 Days
• Field of Search: None
• US Class Current: 709/203
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/151 Transformation
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A method and system for modifying web pages, including dynamic web pages, based on automated analysis wherein web pages are transformed based on transformation instructions in nearly real-time, and wherein analysis is performed and transformation instructions based on the analysis are prepared prior to a request for the web page. The system has two primary components, an analyzer which asynchronously and repeatedly analyzes web pages creating and updating transformation instructions relating to the web pages, and a transformer which intercepts traffic to a web server in response to a request for the web page, receives the returned web pages, and transforms them based on stored transformation instructions.

Claims

1. In a system wherein a web page is accessible to a client from a server, and wherein the web page has an associated Uniform Resource Locator (URL) or link thereto corresponding to an address, wherein in response to a request for the web page the server provides the web page to the client, a method of, providing a modified web page to the client in response to a request from the client, comprising:

asynchronous to, and prior to said request from the client, in dependence upon predetermined conditions, analyzing at least a portion of the requested web page with an analyzer in a computer based system to identify at least a predetermined characteristic and creating transformation instructions that will change the predetermined characteristic when the web page is modified; and

storing the transformation instructions in a repository;

modifying the web page provided by the server in response to the request to the web page based on transformation instructions that were stored in the repository prior to said request from the client; and

,

providing the modified web page to the client.

2. A method as defined in claim 1 wherein transformation instructions are stored in the repository for a plurality of requests for the same web page.

3. A method as defined in claim 2 wherein same transformation instructions stored within the repository are provided to transformers residing in different locations in response to requests for the web page from a plurality of clients.

4. A method as defined in claim 2 wherein at least some transformation instructions stored within the repository are cached proximate to the transformer.

5. A method as defined in claim 1 wherein the step of analyzing includes parsing HTML portions of the web page.

6. A method of claim 5 wherein the step of modifying the web page based on at least a transformation instruction does not require HTML parsing of any portion of the web page.

7. The method of claim 1 wherein the creation of transformation instructions also includes the creation of a resource related to a transformation instruction.

8. A method as defined in claim 7 wherein the created resource is stored on at least a part of a Content Delivery Network (CDN) and wherein a transformation instruction references said resource.

9. A method as defined in claim 1 wherein the analyzer requests the web page from the server prior to analyzing to obtain the web page to analyze.

10. A method as defined in claim 1 wherein the predetermined conditions include the web page being requested for the first time since a predefined point in time.

11. A method as defined in claim 1, wherein the web page includes JavaScript and wherein the step of analyzing includes statically analyzing at least some of the JavaScript on the web page.

12. A method as defined in claim 1 wherein the web page includes some JavaScript and wherein the step of analyzing includes executing at least some of the JavaScript on the web page.

13. A method as defined in claim 1 wherein the analyzer repeatedly monitors the repository for requested web pages to analyze.

14. A system for changing the content of a requested web page, in response to a request for the web page from a client, so as to vary a characteristic thereof comprising:

an analyzer including one or more suitably programmed processors operative asynchronous to, and prior to the request from the client, to analyze at least a portion of the web page to identify at least a predetermined characteristic and for creating transformation instructions corresponding to the characteristic;

a memory to store the transformation instructions received from the analyzer, the transformation instructions being stored prior to the request from the client; and

a transformer operative upon receipt of the request for the web page to modify the web page based on the transformation instructions previously stored in the memory prior to the request for the web page and to return the modified web page to the client in response to the request for the web page.

15. A system as defined in claim 14 wherein the transformer includes software for performing the transformations on one or more processors.

16. A system as defined in claim 14 wherein the analyzer and the transformer utilize different processors to analyze and modify, respectively.

17. A system as defined in claim 16 comprising a plurality of transformers on different servers for receiving transformation instructions from said memory at the first location.

18. A system as defined in claim 14, wherein the analyzer utilizes memory at a first location and wherein the transformer utilizes local cache memory at a second different location proximate to the transformer.

19. A system as defined in claim 14 wherein the analyzer includes software for performing said analysis on the one or more processors.

20. The system as described in claim 14 wherein the transformer operates in a request and response processing flow between the client and an origin server that publishes the web page, and the analyzer operates externally to the request and response processing flow between the client and the origin server.