Automatic migration of data via a distributed computer network

• US 7,143,170 B2
• Filed: 04/29/2004
• Issued: 11/28/2006
• Est. Priority Date: 04/30/2003
• Status: Active Grant

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Akamai Technologies, Inc.
• Inventors: Swildens, Eric Sven-Johan, Cinquini, Maurice, Agarwal, Anshu, Chavarkar, Amol
• Primary Examiner(s): Vu, Viet D.
• Application Number:
  US10/835,974
• Publication Number:
  US 20040221019A1
• Time in Patent Office: 943 Days
• Field of Search: 709/201, 709/203, 709/217, 709/219, 709/220, 709/223, 709/226, 709/227, 709/229, 709/249
• US Class Current: 709/226
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1038 Load balancing arrangements to avoid a single path through a load balancer
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/563 Data redirection of data network streams
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A method and apparatus for the automatic migration of data via a distributed computer network allows a customer to select content files that are to be transferred to a group of edge servers. Origin sites store all of a customer'"'"'s available content files. An edge server maintains a dynamic number of popular files in its memory for the customer. The files are ranked from most popular to least popular and when a file has been requested from an edge server a sufficient number of times to become more popular than the lowest popular stored file, the file is obtained from an origin site. The edge servers are grouped into two service levels: regional and global. The customer is charged a higher fee to store its popular files on the global edge servers compared to a regional set of edge servers because of greater coverage.

Claims

1. A method, comprising:

providing a plurality of edge servers;

wherein edge servers are servers that deliver content to end users;

wherein the edge servers serve a customer'"'"'s most popular content to requesting users;

wherein a customer is allocated a certain amount of memory to store the customer'"'"'s most popular content on each edge server in a set of edge servers, the amount of memory based on a fee paid by the customer;

wherein at least one specific set of edge servers among a plurality of sets of edge servers serve a particular customer'"'"'s most popular content to requesting users;

tracking popularity of a customer'"'"'s content on an edge server based on number of requests for the customer'"'"'s content;

creating an ordered list of a customer'"'"'s most popular content files using number of requests tracked for a particular file; and

evaluating the ordered list for newly popular content files not stored in the edge server'"'"'s local storage that have become more popular than content files stored in the edge server'"'"'s local storage.

2. The method of claim 1, further comprising:

wherein a plurality of origin servers store all of a customer'"'"'s available content; and

replicating newly popular content onto the edge server'"'"'s local storage from content from the origin servers.

3. The method of claim 2, further comprising replacing a lower popularity content file in local storage with a replicated content file.

4. The method of claim 2:

wherein the customer schedules content to be uploaded from a customer server to a staging server; and

wherein the staging server uploads received content to appropriate origin servers.

5. The method of claim 1, wherein the plurality of edge servers are partitioned into a plurality of sets of edge servers based on geographic regions.

6. The method of claim 5, wherein the plurality of edge servers are further partitioned into a global set of edge servers.

7. The method of claim 1, wherein the plurality of edge servers are dynamically partitioned into a plurality of sets of edge servers based on using factors that include any of:

lower current cost, higher performance, and time-based availability.

8. The method of claim 1, further comprising:

receiving a user'"'"'s request for customer content at an edge server;

sending the requested content to the user if the requested content is stored on the edge server'"'"'s local storage; and

redirecting the user to an origin server that is likely to have the content stored locally if the edge server does not have the requested content.

9. The method of claim 8, wherein the redirecting step is located on a server that selects the best origin server by favoring origin servers that should be able to access the content on their local storage device rather than across a network.

10. The method of claim 8, further comprising:

receiving the user'"'"'s redirected request on the origin server;

sending the requested content to the user if the requested content is stored on the origin server'"'"'s local storage; and

redirecting the user to the customer'"'"'s staging server if the origin server does not have the requested content.

11. The method of claim 10, wherein the redirecting step is located on a server that selects the best staging server by favoring staging servers that should be able to access the requested content on their local storage device rather than across a network.

23. The method of claim 1, wherein the plurality of sets of edge servers are dynamically determined using factors that include any of:

lower current cost, higher performance, and time-based availability.

12. An apparatus, comprising:

a plurality of edge servers;

wherein edge servers are servers that deliver content to end users;

wherein the edge servers serve a customer'"'"'s most popular content to requesting users;

wherein a customer is allocated a certain amount of memory to store the customer'"'"'s most popular content on each edge server in a set of edge servers, the amount of memory based on a fee paid by the customer;

wherein at least one specific set of edge servers among a plurality of sets of edge servers serve a particular customer'"'"'s most popular content to requesting users;

tracking means on an edge server for tracking popularity of a customer'"'"'s content based on number of requests for the customer'"'"'s content;

wherein the tracking means creates an ordered list of a customer'"'"'s most popular content files using the number of requests tracked for a particular file; and

a module for evaluating the ordered list for newly popular content files not stored in the edge server'"'"'s local storage that have become more popular than content files stored in the edge server'"'"'s local storage.

13. The apparatus of claim 12, further comprising:

wherein a plurality of origin servers store all of a customer'"'"'s available content; and

replicating means on the edge server for replicating newly popular content onto the edge server'"'"'s local storage from content from the origin servers.

14. The apparatus of claim 13, further comprising a module for replacing a lower popularity content file in local storage with a replicated content file.

15. The apparatus of claim 13:

wherein the customer schedules content to be uploaded from a customer server to a staging server; and

wherein the staging server uploads received content to appropriate origin servers.

16. The apparatus of claim 12, wherein the plurality of edge servers are partitioned into a plurality of sets of edge servers based on geographic regions.

17. The apparatus of claim 16, wherein the plurality of edge servers are further partitioned into a global set of edge servers.

18. The apparatus of claim 12, wherein the plurality of edge servers are dynamically partitioned into a plurality of sets of edge servers based on using factors that include any of:

lower current cost, higher performance, and time-based availability.

19. The apparatus of claim 12, further comprising:

receiving means on an edge server for receiving a user'"'"'s request for customer content;

means for sending the requested content to the user if the requested content is stored on the edge server'"'"'s local storage; and

means for redirecting the user to an origin server that is likely to have the content stored locally if the edge server does not have the requested content.

20. The apparatus of claim 19, wherein the redirecting means is located on a server that selects the best origin server by favoring origin servers that should be able to access the content on their local storage device rather than across a network.

21. The apparatus of claim 19, further comprising:

means for receiving the user'"'"'s redirected request on the origin server;

means for sending the requested content to the user if the requested content is stored on the origin server'"'"'s local storage; and

means for redirecting the user to the customer'"'"'s staging server if the origin server does not have the requested content.

22. The apparatus of claim 21, wherein the redirecting means is located on a server that selects the best staging server by favoring staging servers that should be able to access the requested content on their local storage device rather than across a network.

wherein the plurality of edge servers are further partitioned into a global set of edge servers, and wherein the charging step charges the customer a higher fee for a memory allocation in the global set of edge servers.

24. The apparatus of claim 12, wherein the plurality of sets of edge servers are dynamically determined using factors that include any of:

lower current cost, higher performance, and time-based availability.