Optimized retrieval of network resources

• US 9,722,851 B1
• Filed: 03/27/2012
• Issued: 08/01/2017
• Est. Priority Date: 03/27/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Hill, Peter F.
• Primary Examiner(s): Nguyen, Quang N
• Application Number:
  US13/431,687
• Time in Patent Office: 1,953 Days
• Field of Search: 709217-219, 709203, 709206, 709223, 709224, 709226, 709228, 709231, 709232, 709238
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

Features are described with regard to the analysis of network and content characteristics that may affect the retrieval of network content, and the determination by a network computing device, based at least partly on that analysis, of whether a client computing device should bypass the network computing device and retrieve requested and related content directly from content sources. Additional features are described with regard to the determination of network resources to be cached, and to the determination of a computing device from which to initiate the caching. The network computing component or the client computing device can also monitor characteristics of the network connection between the computing devices and with other computing devices associated with content providers.

Claims

1. A system for managing retrieval of web pages, the system comprising:

a network computing component comprising one or more computing devices, the network computing component configured to;

determine that a probability, of a web page being subsequently requested by a client computing device, satisfies a threshold, wherein the web page is hosted by a content provider separate from the network computing component, and wherein the network computing component is not the content provider;

obtain profile information regarding the web page, the profile information specifying whether the client computing device should retrieve the web page from the content provider;

determine, based at least in part on the profile information, whether to retrieve the web page from the content provider on behalf of the client computing device instead of the client computing device retrieving the web page from the content provider;

in response to determining that the client computing device should retrieve the web page from the content provider, transmit a notification to the client computing device, the notification indicating that the client computing device is to retrieve the web page from the content provider; and

in response to determining to retrieve the web page from the content provider on behalf of the client computing device, retrieve the web page from the content provider and process the web page to generate a processed version of the web page.

2. The system of claim 1, wherein the network computing component is further configured to obtain data regarding one or more characteristics of a network connection to the client computing device, and wherein determining whether the client computing device should retrieve the web page from the content provider is further based at least in part on the one or more characteristics of the network connection.

3. The system of claim 2, wherein the one or more characteristics comprise network latency.

4. The system of claim 2, wherein the one or more characteristics comprise geographic distance between the client computing device and the content provider.

5. A computer-implemented method for managing retrieval of network resources, the computer-implemented method comprising:

determining, by a network computing component comprising one or more computing devices configured to execute specific instructions, that a probability of a network resource being subsequently requested by a client computing device satisfies a threshold, wherein the network resource is hosted by a content source separate from the network computing component, and wherein the network computing component is not the content source;

obtaining, by the network computing component, profile information associated with the network resource;

determining, based at least in part on the profile information, whether the client computing device should retrieve the network resource from the content source instead of retrieving the network resource from the content source for the client computing device;

in response to determining that the client computing device should retrieve the network resource from the content source, transmitting a notification to the client computing device to retrieve the network resource from the content source; and

in response to determining to retrieve the network resource from the content source on behalf of the client computing device, retrieving the network resource from the content source and processing the network resource to generate a processed version of the network resource.

6. The computer-implemented method of claim 5, wherein the network resource is a web page.

7. The computer-implemented method of claim 5, wherein the profile information comprises information regarding optimized retrieval of the network resource.

8. The computer-implemented method of claim 5, wherein the profile information specifies that a requesting client computing device is to retrieve the network resource from the content source.

9. The computer-implemented method of claim 5, further comprising:

obtaining, by the network computing component, data regarding one or more characteristics of a network connection to the client computing device,

wherein determining whether the client computing device should retrieve the network resource from the content source is further based at least in part on the data.

10. The computer-implemented method of claim 9, wherein the one or more characteristics comprise network latency.

11. The computer-implemented method of claim 9, wherein the one or more characteristics comprise geographic distance between two endpoints.

12. The computer-implemented method of claim 9, further comprising:

obtaining, by the network computing component, second data regarding one or more characteristics of a network connection to the network computing component,

wherein determining whether the client computing device should retrieve the network resource from the content source is further based at least in part on the second data.

13. A system for managing retrieval of network resources, the system comprising:

a network computing component comprising one or more computing devices, the network computing component configured to;

determine that a probability of a network resource being requested by a client computing device satisfies a threshold, wherein the network resource is hosted by a content source separate from the network computing component, and wherein the network computing component is not the content source;

obtain network characteristic data regarding one or more characteristics of a network connection to the client computing device;

determine, based at least in part on the network characteristic data, whether the client computing device should retrieve the network resource from the content source instead of retrieving the network resource from the content source for the client computing device;

in response to determining that the client computing device should retrieve the network resource from the content source, transmit a notification to the client computing device, the notification indicating that the client computing device is to retrieve the network resource from the content source; and

in response to determining to retrieve the network resource from the content source on behalf of the client computing device, retrieve the network resource from the content source and process the network resource to generate a processed version of the network resource.

14. The system of claim 13, wherein the one or more characteristics comprise network latency.

15. The system of claim 13, wherein the one or more characteristics comprise geographic distance between two endpoints.

16. The system of claim 13, wherein the network computing component is further configured to:

obtain profile information regarding the network resource, wherein the profile information comprises information regarding optimized retrieval of the network resource,

wherein determining whether the client computing device should retrieve the network resource from the content source is further based at least in part on the profile information.

17. The system of claim 16, wherein the profile information specifies that a requesting client computing device should retrieve the network resource from the content source.

18. A computer-implemented method for caching a network resource, the method, as implemented by a network computing component configured with specific executable instructions, comprising:

receiving, from a client computing device, a request for a network resource hosted by a content provider separate from the network computing component;

storing a request record comprising information about the request;

analyzing a plurality of request records to determine a probability that a predicted network resource, different than the network resource, will be requested by the client computing device;

determining that the probability is greater than a threshold probability;

determining, based on network characteristics, whether the client computing device can retrieve the predicted network resource faster than the network computing component can retrieve the predicted network resource and transmit the predicted network resource to the client computing device;

in response to determining to retrieve the predicted network resource from the content provider on behalf of the client computing device, retrieve the predicted network resource from the content provider and process the predicted network resource to generate a processed response to the request; and

in response to determining that the client computing device can retrieve the predicted network resource faster than the network computing component can retrieve the predicted network resource and transmit the predicted network resource to the client computing device, notifying the client computing device to retrieve the predicted network resource.

19. The computer-implemented method of claim 18, wherein the network resource and predicted network resource are web pages.

20. The computer-implemented method of claim 18, wherein network characteristics comprise network latency.

21. The computer-implemented method of claim 18, wherein network characteristics comprise geographic distance between two endpoints.

22. The computer-implemented method of claim 18, wherein the probability of being requested is based at least in part on the content of the network resource.

23. The computer-implemented method of claim 18, wherein the probability of being requested varies based at least in part on the time of the day.

24. The computer-implemented method of claim 18, wherein the probability of being requested is based at least on the number of times the network resource and a second network resource have been requested over the same time period.

25. The computer-implemented method of claim 18, wherein determining the predicted network resource is based on identifying popular network resources that have been requested a greater number of times over a time period than an average of number of requests for a network resource over the time period.

26. A system for managing computing requests, the system comprising:

a network computing component comprising one or more computing devices, the network computing component configured to;

determine that a probability of a network resource being requested by a client computing device satisfies a threshold, wherein the network computing device does not host the network resource;

obtain network characteristic data regarding one or more characteristics of a first network connection of the client computing device to a content provider of the network resource and a second network connection of the client computing device to a second client computing device, wherein the network characteristic data comprises data regarding a first latency of the first network connection and a second latency of the second network connection;

determine, based at least in part on the network characteristic data, which of the following actions to take;

(1) obtain the network resource on behalf of the client computing device from the content provider and transmit the network resource to the client computing device, or (2) notify the client computing device to obtain the network resource from the second client computing device;

in response to determining to notify the client computing device to obtain the network resource from the second client computing device, transmit a notification to the client computing device; and

in response to determining to retrieve the network resource from the content provider on behalf of the client computing device, retrieve the network resource from the content provider and process the network resource to generate a processed version of the network resource.

27. The system of claim 26, wherein the notification instructs the client computing device to establish a peer to peer connection with the second client computing device to retrieve the network resource.

28. The system of claim 26, wherein the network characteristic data comprises data regarding at least one of the following:

the first latency of the network connection of the client computing device to the content provider;

the second latency of the network connection of the client computing device to the second client computing device;

a historical latency of the second network connection of the second client computing device;

or a geographic distance between the client computing device and the second client computing device.

29. The system of claim 26, wherein the network computing device is further configured to obtain data regarding a third latency of a third network connection of the client computing device to another content provider of the network resource and a fourth latency of a fourth network connection of the network computing device to the other content provider.

30. The system of claim 26, wherein the network computing device is further configured to:

access historical information regarding the first latency of the first network connection;

determine the first latency of the first network connection based at least in part on the historical information or a ping from the network computing device to the content provider;

determine a geographic distance of the second network connection based on a first IP address associated with the client computing device and a second IP address associated with the second client computing device; and

determine the second latency of the second network connection based at least in part on the geographic distance of the second network connection.