LOADING OF WEB RESOURCES

• US 20130041946A1
• Filed: 10/09/2012
• Published: 02/14/2013
• Est. Priority Date: 05/20/2011
• Status: Active Grant

• Current Assignee: CloudFlare Incorporated
• Original Assignee: CloudFlare Incorporated
• Inventors: Joel, Christopher Stephen, Benterou, Jason Thomas Walter, Holloway, Lee Hahn, Prince, Matthew Browning, Pye, Ian Gerald
• Granted Patent: US 9,342,620 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]

Abstract

A method and apparatus for improving loading of web resources. A server receives a request for a Hypertext Markup Language (HTML) document requested by a client network application. The server retrieves the requested document. The server automatically modifies objects referenced in the HTML document that have an external source such that loading of those objects by the client network application will be deferred. The server inserts a client-side script loader or a reference to the client-side script loader into the HTML document. The client-side script loader is configured to, when executed by the client network application, attempt to load the objects that have been deferred. The server transmits the modified HTML document to the client network application.

Claims

1. A method in a server for improving the loading of web resources, the method comprising:

receiving a request from a client network application for a Hypertext Markup Language (HTML) document, wherein the request identifies a domain that hosts the HTML document;

retrieving the requested document;

automatically modifying a plurality of objects referenced in the HTML document that have an external source such that loading of the plurality of objects by the client network application will be deferred;

inserting a client-side script loader or a reference to the client-side script loader into the HTML document, wherein the client-side script loader is configured to, when executed by the client network application, attempt to load the plurality of objects; and

transmitting the modified HTML document to the client network application.

2. The method of claim 1, wherein the server is a proxy server, and wherein the request is received at the proxy server as a result of a Domain Name System (DNS) request for the domain resolving to the proxy server.

3. The method of claim 1, wherein the client-side script loader is configured to be executed by the client network application only after the client network application has finished loading HTML of the HTML document other than the modified objects.

4. The method of claim 1, wherein the client-side script loader is configured to, when executed by the client network application, establish a Transmission Control Protocol (TCP) connection with the server and transmit a single request to the server for a set of one or more objects using that TCP connection.

5. The method of claim 4, further comprising:

establishing a single Transmission Control Protocol (TCP) connection with the client network application;

receiving a single request over the single TCP connection from the client network application for the set of one or more objects;

attempting to retrieve each of the set of objects requested in the single request;

responsive to retrieving a first one of the set of objects, forming a first response that includes the retrieved object and transmitting that first response to the client network application using the single TCP connection, wherein each subsequent one of the set of objects that is retrieved is included in the first response and streamed to the client network application asynchronously.

6. The method of claim 5, wherein attempting to retrieve each of the set of objects includes performing the following for each of the set of objects:

determining whether that object is in cache and has not expired;

responsive to determining that that object is in the cache and has not expired, retrieving that object from the cache; and

responsive to determining that that object is not in cache or is in cache but expired, performing the following;

transmitting a request to an origin server that hosts that object, wherein responsive to receiving a second response with that object from the origin server, caching that object in the cache and including that object in the first response.

7. The method of claim 6, wherein the client-side script loader is configured to request a standard package of objects for the set of objects, wherein the standard package of objects is a set of one or more objects that are historically known to be included for the HTML document.

8. The method of claim 6, wherein the set of objects include those of the plurality of objects referenced in the HTML document that are not locally available to the client network application.

9. A non-transitory machine-readable storage medium that provides instructions that, when executed by a processor of a server, cause said processor to perform operations comprising:

receiving a request from a client network application for a Hypertext Markup Language (HTML) document, wherein the request identifies a domain that hosts the HTML document;

retrieving the requested document;

automatically modifying a plurality of objects referenced in the HTML document that have an external source such that loading of the plurality of objects by the client network application will be deferred;

inserting a client-side script loader or a reference to the client-side script loader into the HTML document, wherein the client-side script loader is configured to, when executed by the client network application, attempt to load the plurality of objects; and

transmitting the modified HTML document to the client network application.

10. The non-transitory machine-readable storage medium of claim 9, wherein the server is a proxy server, and wherein the request is received at the proxy server as a result of a Domain Name System (DNS) request for the domain resolving to the proxy server.

11. The non-transitory machine-readable storage medium of claim 9, wherein the client-side script loader is configured to be executed by the client network application only after the client network application has finished loading HTML of the HTML document other than the modified objects.

12. The non-transitory machine-readable storage medium of claim 9, wherein the client-side script loader is configured to, when executed by the client network application, establish a Transmission Control Protocol (TCP) connection with the server and transmit a single request to the server for a set of one or more objects using that TCP connection.

13. The non-transitory machine-readable storage medium of claim 12, further to provide instructions that, when executed by the processor of the server, cause the processor to perform operations comprising:

establishing a single Transmission Control Protocol (TCP) connection with the client network application;

receiving a single request over the single TCP connection from the client network application for the set of one or more objects;

attempting to retrieve each of the set of objects requested in the single request;

responsive to retrieving a first one of the set of objects, forming a first response that includes the retrieved object and transmitting that first response to the client network application using the single TCP connection, wherein each subsequent one of the set of objects that is retrieved is included in the first response and streamed to the client network application asynchronously.

14. The non-transitory machine-readable storage medium of claim 13, wherein attempting to retrieve each of the set of objects includes performing the following for each of the set of objects:

determining whether that object is in cache and has not expired;

responsive to determining that that object is in the cache and has not expired, retrieving that object from the cache; and

responsive to determining that that object is not in cache or is in cache but expired, performing the following;

transmitting a request to an origin server that hosts that object, wherein responsive to receiving a second response with that object from the origin server, caching that object in the cache and including that object in the first response.

15. The non-transitory machine-readable storage medium of claim 14, wherein the client-side script loader is configured to request a standard package of objects for the set of objects, wherein the standard package of objects is a set of one or more objects that are historically known to be included for the HTML document.

16. The non-transitory machine-readable storage medium of claim 14, wherein the set of objects include those of the plurality of objects referenced in the HTML document that are not locally available to the client network application.

17. An apparatus for improving the loading of a web page, the apparatus comprising:

a memory configured to store instructions that, when executed by a processor, cause said processor to perform the following;

retrieve a Hypertext Markup Language (HTML) document requested by a client network application;

automatically modify a plurality of objects referenced in the HTML document that have an external source such that loading of the plurality of objects by the client network application will be deferred;

insert a client-side script loader or a reference to the client-side script loader into the HTML document, wherein the client-side script loader is configured to, when executed by the client network application, attempt to load the plurality of objects; and

transmit the modified HTML document to the client network application.

18. The apparatus of claim 17, wherein the apparatus is a proxy server, and wherein the request is received at the proxy server as a result of a Domain Name System (DNS) request for the domain resolving to the proxy server.

19. The apparatus of claim 17, wherein the client-side script loader is configured to be executed by the client network application only after the client network application has finished loading HTML of the HTML document other than the modified objects.

20. The apparatus of claim 17, wherein the client-side script loader is configured to, when executed by the client network application, establish a Transmission Control Protocol (TCP) connection with the apparatus and transmit a single request to the apparatus for a set of one or more objects using that TCP connection.

21. The apparatus of claim 20, wherein the memory is configured further to store instructions, that when executed by the processor, cause said processor to perform the following:

establish a single Transmission Control Protocol (TCP) connection with the client network application;

receive a single request over the single TCP connection from the client network application for the set of one or more objects;

attempt to retrieve each of the set of objects requested in the single request;

responsive to retrieval a first one of the set of objects, form a first response that includes the retrieved object and transmit that first response to the client network application using the single TCP connection, wherein each subsequent one of the set of objects that is retrieved is included in the first response and streamed to the client network application asynchronously.

22. The apparatus of claim 21, wherein the attempt to retrieve each of the set of objects includes the processor to perform the following for each of the set of objects:

determine whether that object is in cache and has not expired;

responsive to a determination that that object is in the cache and has not expired, retrieve that object from the cache; and

responsive to a determination that that object is not in cache or is in cache but expired, transmit a request to an origin server that hosts that object, wherein responsive to receipt of a second response with that object from the origin server, cache that object in the cache and include that object in the first response.

23. The apparatus of claim 22, wherein the client-side script loader is configured to request a standard package of objects for the set of objects, wherein the standard package of objects is a set of one or more objects that are historically known to be included for the HTML document.

24. The apparatus of claim 22, wherein the set of objects include those of the plurality of objects referenced in the HTML document that are not locally available to the client network application.