Method and apparatus for managing a plurality of servers in a content delivery network

• US 7,181,523 B2
• Filed: 05/18/2001
• Issued: 02/20/2007
• Est. Priority Date: 10/26/2000
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Sim, Siew Yong
• Primary Examiner(s): VU, VIET DUY
• Application Number:
  US09/681,669
• Publication Number:
  US 20020083118A1
• Time in Patent Office: 2,104 Days
• Field of Search: 709/201, 709/202, 709/203, 709/220, 709/223, 709/224, 709/226, 718/102, 718/104, 718/105
• US Class Current: 709/226
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1019 Random or heuristic server selection
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1027 Persistence of sessions during load balancing
• CPC Class Codes: H04L 67/1034 Reaction to server failures by a load balancer
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/14 Multichannel or multilink protocols
• CPC Class Codes: Y02D 30/50 in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

Large payload files are selectively partitioned in blocks and the blocks distributed to a plurality of distribution stations at the edge of the network qualified to have the data. Each qualified station decides how much and what portion of the content to save locally, based on information such as network location and environment, usage, popularity, and other distribution criteria defined by the content provider. Different pieces of a large payload file may be available from different nodes, however, when a user requests access to the large payload file, for example, through an application server, a virtual file control system creates an illusion that the entire file is present at the connected node. However, since only selective portions of the large payload file may actually be resident at that node'"'"'s storage at the time of request, a cluster of distribution servers at the distribution station may download the non-resident portions of the file as the application server is servicing the user. The download may be in parallel and usually from the least congested nodes. New nodes added to the network learn from other nodes in the network what content they should have and download the required content, in a desired amount, onto their local storage devices from the nearest and least congested nodes without interrupting network operation. Each node manages its local storage and decides what content to prune based on information such as usage patterns.

Claims

1. A method comprising:

distributing content to multiple nodes in a network having a plurality of nodes arranged in the form of a virtual tree, wherein each node in the virtual tree stores a portion of the content and is configured to appear as if all of the content is stored locally at the node;

distributing the content as one or more incoming tasks to one or more available servers in a first network node having a plurality of servers capable of processing the one or more incoming tasks; and

maintaining a list of tasks including all tasks from the one or more incoming tasks distributed for each of the one or more available servers.

2. The method of claim 1, further comprising redistributing all tasks of a first server of the plurality of servers that becomes designated as an unavailable server among the one or more available servers.

3. The method of claim 2, wherein each of one or more available servers monitor a heartbeat of other available servers and uses the heartbeat to determine when at least one server of the plurality of servers becomes designated as the unavailable server.

4. The method of claim 1, wherein the all tasks are treated as jobs to be executed and the list of tasks comprises a job queue.

5. The method of claim 1, wherein the one or more available servers vote for a master server at power up.

6. The method of claim 1, wherein each one of the one or more available servers registers a heartbeat in a database.

7. The method of claim 5, wherein if the unavailable server is the master server, the one or more available servers vote for a new master server.

8. The method of claim 5, wherein the distributing the all tasks from the list of tasks is performed by the master server.

9. A method comprising:

distributing content to multiple nodes in a network having a plurality of nodes arranged in the form of a virtual tree, wherein each node in the virtual tree stores a portion of the content and is configured to appear as if all of the content is stored locally at the node;

distributing the content as one or more incoming tasks to one or more available servers in a first network node having a plurality of servers capable of processing the one or more incoming tasks, wherein the one or more available servers comprises a functioning subset of the plurality of servers; and

maintaining a job queue including all tasks from the one or more incoming tasks distributed to the one or more available servers; and

the master server redistributing all jobs from the job queue of each of the plurality of servers that becomes designated as an unavailable server to the one or more available servers.

10. The method of claim 9, further comprising selecting a master server amongst the plurality of servers in the first network node.

11. The method of claim 10, wherein selecting the master server is performed by the plurality of servers.

12. The method of claim 9, further comprising:

each of the one or more available servers monitoring a heartbeat of other available servers; and

using the heartbeat to determine when at least one server of the plurality of servers becomes designated as the unavailable server.

13. A computer program product comprising:

a computer usable medium comprising computer readable code for managing a plurality of servers in a content delivery network, the computer readable program code configured to;

distribute content to multiple nodes in a network having a plurality of nodes arranged in the form of a virtual tree, wherein each node in the virtual tree stores a portion of the content and is configured to appear as if all of the content is stored locally at the node;

distribute the content as one or more incoming tasks to one or more available servers in a first network node having a plurality of servers capable of processing the one or more incoming tasks; and

maintain a list of tasks including all tasks from the one or more incoming tasks distributed for each of the one or more available servers.

14. The computer readable program code of claim 13 further configured to redistribute all tasks of a first server of the plurality of servers that becomes designated as an unavailable server among the one or more available servers.

15. The computer readable program code of claim 14, wherein each of one or more available servers monitor a heartbeat of other available servers and uses the heartbeat to determine when at least one server of the plurality of servers becomes designated as the unavailable server.

16. The computer readable program code of claim 13, wherein the all tasks are treated as jobs to be executed and the list of tasks comprises a job queue.

17. The computer readable program code of claim 13, wherein the one or more available servers vote for a master server at power up.

18. The computer readable program code of claim 13, wherein each one of the one or more available servers registers a heartbeat in a database.

19. The computer readable program code of claim 18, wherein if the unavailable server is the master server, the one or more available servers vote for a new master server.

20. The computer readable program code of claim 18, wherein the distributing the all tasks from the list of tasks is performed by the master server.

21. A network comprising:

a first network node; and

a second network node, wherein the first and second network nodes are arranged in the form of a virtual tree, store a portion of content and are configured to appear as if all of the content is stored locally at each node, wherein each node includes;

a plurality of servers capable of performing identical tasks;

a load balancer to distribute one or more incoming tasks to one or more available servers in the network node, wherein, each of the plurality of servers maintain a list of tasks including the distributed incoming tasks;

a master server to redistribute to the one or more available servers all tasks from the list of tasks of a server of the plurality of servers that becomes designated as an unavailable server.

22. The network of claim 21, wherein the load balancer redistributes all tasks of a first server of the plurality of servers that becomes designated as an unavailable server among the one or more available servers.

23. The network of claim 22, wherein each of one or more available servers monitor a heartbeat of other available servers and uses the heartbeat to determine when at least one server of the plurality of servers becomes designated as the unavailable server.

24. The network of claim 21, wherein the tasks are treated as jobs to be executed and the list of tasks comprises a job queue.

25. The network of claim 21, wherein the one or more available servers vote for the master server at power up.