Providing faster data access using multiple caching servers

• US 10,021,210 B1
• Filed: 11/16/2017
• Issued: 07/10/2018
• Est. Priority Date: 03/24/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Joseph, Infant Gerald Prasath
• Primary Examiner(s): Sall, El Hadji
• Application Number:
  US15/815,637
• Time in Patent Office: 236 Days
• Field of Search: 709213, 709203, 709200, 709226
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A method and system for identifying an optimal server to receive requests for network content requested by a user of a network device is provided. A browser application in a network device receives a request for network content from a user and transmits the request to a server. The browser application receives the network content from the server and renders the network content to the user on the network device. Executable code in the rendered network content enables the browser application to identify an optimal server to receive subsequent items of network content requested by the user. When the user selects an item of network content in the rendered network page, the browser application connects to the optimal server to receive subsequent items of network content for the user.

Claims

1. A computer-implemented method comprising:

in response to receiving a request for an item of network content from a network client, identifying a plurality of servers having access to the network content, the plurality of servers comprising at least a base server and one or more cache servers;

generating a content request for the item of network content;

transmitting the content request for the item of network content to individual servers of the plurality of servers in parallel;

receiving, from at least a subset of the plurality of servers, a plurality of responses to the content request that comprises at least a portion of the item of network content;

determining, for individual responses in the plurality of responses, a round trip latency for the content request based at least in part on an amount of time to receive the response;

determining that an optimal server of the plurality of servers is associated with a received response having the lowest round trip latency of the plurality of responses; and

determining a lock-in period for the network client, wherein one or more subsequent requests received from the network client are transmitted to the optimal server during the lock-in period.

2. The computer-implemented method of claim 1, wherein the lock-in period is determined based at least in part on historical performance data related to the optimal server.

3. The computer-implemented method of claim 2, wherein the historical performance data is obtained by probing the optimal server at fixed-intervals of time to determine if there are updates to performance data related to the optimal server.

4. The computer-implemented method of claim 1, wherein identifying the plurality of servers is based on at least one of a geographical location of individual servers of the plurality of servers, server load information, or prior static knowledge of hop distances to the plurality of servers.

5. The computer-implemented method of claim 1, further comprising:

receiving, from the optimal server, a second plurality of servers that includes at least one cache server not included in the plurality of servers; and

in response to detecting an expiration of the lock-in period, determining a new lock-in period for the network client with respect to a second optimal server of the second plurality of servers.

6. The computer-implemented method of claim 5, wherein the second plurality of servers is identified by the optimal server based on at least one of a geographical location of each of the second plurality of servers, server load information, or prior static knowledge of hop distances to the second plurality of servers.

7. A system, comprising:

a memory that stores computer-executable instructions; and

at least one processor configured to access the memory, the at least one processor configured to execute the computer-executable instructions to collectively at least;

receive a first request for an item of network content from a browser application installed on a network client;

transmit, to a plurality of servers including a base sever and at least one cache server, a plurality of second requests for the item;

receive, from at least a subset of the plurality of servers, at least two responses;

determine, for individual responses of the at least two responses, a round trip latency for the response;

identify, based on the determined round trip latency, an optimal server of the subset of the plurality of servers corresponding with a shortest round trip latency; and

instruct the browser application on the network client to transmit at least a third request for a subsequent item of network content to the optimal server for a determined lock-in period.

8. The system of claim 7, wherein the round trip latency for individual responses of the at least two responses is determined based on information included in the response.

9. The system of claim 7, wherein the round trip latency for individual responses of the at least two responses is determined based on a first time at which the plurality of second requests are transmitted and a second time at which each of the at least two responses are received.

10. The system of claim 7, wherein at least one of the at least two responses is received from the at least one cache server, and wherein at least another of the at least two responses is received from the base server.

11. The system of claim 7, wherein the round trip latency for individual responses of the at least two responses is stored as cookie information at the network client with respect to the browser application.

12. The system of claim 7, wherein the at least a third request includes all requests received from the network client during the lock-in period.

13. The system of claim 7, wherein the lock-in period for the optimal server is dynamically determined based on at least one of past web page load times experienced by the optimal server, the typical number of requests per second handled by the optimal server, or the typical load time experienced by the optimal server at various load levels.

14. A computer apparatus, comprising:

a processor; and

a memory that stores computer-executable instructions, the computer-executable instructions configured to, when executed by the processor, cause the computer apparatus to;

receive, from a network client, a request for an item of network content;

identify a plurality of servers having access to the network content, the plurality of servers comprising at least a base server and one or more cache servers;

generate a content request for the item of network content;

transmit the content request for the item of network content to individual servers of the plurality of servers in parallel;

receive, from at least a subset of the plurality of servers, a plurality of responses to the content request that comprises at least a portion of the item of network content;

determine, for individual responses in the plurality of responses, a round trip latency for the content request based at least in part on an amount of time to receive the response;

determine that an optimal server of the plurality of servers is associated with a received response having a lowest round trip latency of the plurality of responses;

identify a lock-in period associated with the optimal server; and

cause one or more subsequent requests received from the network client to be transmitted to the optimal server during the lock-in period.

15. The computer apparatus of claim 14, wherein the request for the item of network content is a search query received via a search engine.

16. The computer apparatus of claim 14, wherein the request for the item of network content is a web address received via an address bar.

17. The computer apparatus of claim 14, wherein the content request is a light-weight probe, and wherein the portion of the item of network content includes only a first several bytes of the item of network content.

18. The computer apparatus of claim 17, wherein the computer-executable instructions are further configured to cause the computer apparatus to submit a second content request to the optimal server to obtain the complete item of network content.

19. The computer apparatus of claim 18, wherein the second content request is submitted to the optimal server upon receiving an indication that a user of the network client has selected an indicator of the portion of the item of network content rendered on a webpage.

20. The computer apparatus of claim 14, wherein the item of network content is a webpage requested by a user of the network client.