System and method for efficiently replicating a file among a plurality of recipients in a reliable manner

• US 7,512,701 B2
• Filed: 01/16/2003
• Issued: 03/31/2009
• Est. Priority Date: 01/16/2003
• Status: Expired due to Fees

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Cherkasova, Ludmila
• Primary Examiner(s): Dalencourt; Yves
• Application Number:
  US10/345,587
• Publication Number:
  US 20040143647A1
• Time in Patent Office: 2,266 Days
• Field of Search: 709/217, 709/219, 709/231, 709/236, 709/238, 709/248
• US Class Current: 709/236
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: G06F 8/61 Installation
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

In accordance with one embodiment, a method of distributing a file from a first node to a plurality of recipient nodes comprises partitioning a file F into a plurality of subfiles. The method further comprises performing distribution of the file F to a plurality of recipient nodes using a distribution technique that comprises (a) attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes, and (b) the plurality of recipient nodes of the first group attempting to exchange their respective subfiles received from the first node. The method further comprises detecting a failed node, and the distribution technique adapting to distribute the file F to each non-failed node of the plurality of recipient nodes.

Claims

1. A method of distributing a file from a first node to a plurality of recipient nodes, the method comprising:

partitioning a file F into a plurality of subfiles;

performing distribution of said file F to a plurality of recipient nodes using a distribution technique that comprises

(a) attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes, wherein the first node attempts to communicate at least one subfile to each recipient node of said first group but not all of said plurality of subfiles to any recipient node of said first group, and

(b) said plurality of recipient nodes of said first group attempting to exchange their respective subfiles received from said first node;

detecting a failed node; and

said distribution technique to distribute said file F to each non-failed node of said plurality of recipient nodes.

2. The method of claim 1 wherein said distribution technique is responsive to said detecting a failed node.

3. The method of claim 1 wherein said attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes comprises:

attempting to distribute a different subfile from said first node to each of said recipient nodes of said first group.

4. The method of claim 1 further comprising:

determining a number of said subfiles to partition said file into.

5. The method of claim 4 wherein said determining comprises:

determining a suitable number k of concurrent communication connections supported by any one of the nodes for concurrent communication with a plurality of the other nodes; and

determining said number of subfiles as corresponding in number to said number k of concurrent communication connections.

6. The method of claim 1 further comprising:

determining a number of said recipient nodes to include in said first group.

7. The method of claim 6 wherein said determining comprises:

determining a suitable number k of concurrent communication connections supported by any one of the nodes for concurrent communication with a plurality of the other nodes; and

determining said number of recipient nodes to include in said first group as corresponding in number to said number k of concurrent communication connections.

8. The method of claim 1 wherein said attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes comprises:

attempting to distribute the plurality of subfiles from said first node to said plurality of recipient nodes of said first group concurrently.

9. The method of claim 1 wherein said plurality of recipient nodes of said first group attempting to exchange their respective subfiles further comprises:

each of said plurality of recipient nodes attempting to establishing concurrent communication connections to every other recipient node of said first group.

10. The method of claim 1 wherein said detecting a failed node comprises said first node detecting a failed node in said first group such that said first node is unable to communicate a particular subfile to such failed node.

11. The method of claim 10 wherein said attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes comprises said first node attempting to establish concurrent communication connections to the recipient nodes of said first group, and wherein said distribution technique comprises:

responsive to said first node detecting a failed node in said first group such that said first node is unable to communicate a particular subfile to such failed node, said first node using the established concurrent communication connections with non-failed nodes of said first group to communicate the particular subfile to said non-failed nodes.

12. The method of claim 1 wherein said detecting a failed node comprises said first node detecting a failed node in said first group after said first node has communicated a particular subfile to such failed node but before such failed node has communicated said particular subfile to every other recipient node of said first group.

13. The method of claim 12 wherein said distribution technique comprises:

responsive to said first node detecting a failed node in said first group after said first node has communicated a particular subfile to such failed node but before such failed node has communicated said particular subfile to every other recipient node of said first group, said first node establishing concurrent communication connections with non-failed nodes of said first group that have not received the particular subfile from said failed node, and said first node communicating the particular subfile to said non-failed nodes.

14. The method of claim 12 wherein said detecting a failed node comprises:

said recipient nodes of said first group exchanging heartbeat messages;

at least one recipient node of said first group detecting a failed node from analysis of heartbeat messages received; and

said at least one recipient node of said first group notifying said first node of said detected failed node.

15. The method of claim 1 further comprising:

said first group of recipient nodes attempting to communicate said file F to a second group comprising a plurality of recipient nodes.

16. The method of claim 15 further comprising:

each recipient node of said first group attempting to communicate a subfile to every recipient node of said second group such that said recipient nodes of said second group each receive all of said plurality of subfiles comprising file F.

17. The method of claim 15 further comprising:

each recipient node of said first group attempting to communicate the subfile that the recipient node received from said first node to every node of the second group.

18. The method of claim 17 wherein each recipient node of said first group attempts to communicate the subfile that the recipient node received from said first node to every node of the second group concurrently.

19. The method of claim 17 wherein said detecting a failed node comprises detecting a failed node in said first group when said nodes of said first group are attempting to communicate said file F to the second group.

20. The method of claim 17 wherein if a recipient node of said first group is detected as failed, then said distribution technique comprises:

each non-failed recipient node of said first group communicating the subfile that it the non-failed recipient node received from said first node to every node of the second group; and

each non-failed recipient node of said first group communicating the subfile that the failed node received from said first node to a corresponding node of the second group.

21. The method of claim 20 wherein said distribution technique further comprises:

a recipient node of said first group communicating the subfile that the failed node received from said first node to a node of the second group corresponding to the failed node of the first group.

48. The method of claim 1 wherein said plurality of recipient nodes of said first group attempting to exchange their respective subfiles received from said first node comprises:

said plurality of recipient nodes of said first group attempting to exchange their respective subfiles received from said first node such that each of said plurality of recipient nodes of said first group obtains all of said plurality of subfiles.

22. A system comprising:

means for partitioning a file F into a plurality of subfiles;

an origin node comprising means for performing distribution of said file F to a plurality of recipient nodes using a distribution technique that comprises

(a) attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes, wherein the first node attempts to communicate at least one subfile to each recipient node of said first group but not all of said plurality of subfiles to any recipient node of said first group, and

(b) said plurality of recipient nodes of said first group attempting to exchange their respective subfiles received from said first node;

means for detecting a failed node; and

said distribution technique, responsive to the detecting means detecting a failed node, to distribute said file F to each non-failed node of said plurality of recipient nodes.

23. The system of claim 22 wherein said attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes comprises:

attempting to distribute a different subfile from said first node to each of said recipient nodes of said first group.

24. The system of claim 22 wherein said attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes comprises:

attempting to distribute the plurality of subflles from said first node to said plurality of recipient nodes of said first group concurrently.

25. The system of claim 22 wherein said plurality of recipient nodes of said first group attempting to exchange their respective subfiles further comprises:

each of said plurality of recipient nodes attempting to establishing concurrent communication connections to every other recipient node of said first group.

26. The system of claim 22 wherein said detecting means comprises:

means for said first node detecting a failed node in said first group.

27. The system of claim 26 wherein said means for said first node detecting a failed node in said first group comprises:

means for said first node detecting that said first node is unable to communicate a particular subfile to a node of said first group.

28. The system of claim 26 wherein said attempting to distribute the plurality of subfiles from a first node to a first group of recipient nodes comprises said first node attempting to establish concurrent communication connections to the recipient nodes of said first group, and wherein said distribution technique adapting comprises:

responsive to said means for said first node detecting a failed node in said first group such that said first node is unable to communicate a particular subfile to such failed node, said first node using the established concurrent communication connections with non-failed nodes of said first group to communicate the particular subfile to said non-failed nodes.

29. The system of claim 22 wherein said detecting means comprises:

means for said first node detecting a failed node in said first group after said first node has communicated a particular subfile to such failed node but before such failed node has communicated said particular subfile to every other recipient node of said first group.

30. The system of claim 29 wherein said distribution technique adapting comprises:

responsive to said means for said first node detecting a failed node in said first group after said first node has communicated a particular subfile to such failed node but before such failed node has communicated said particular subfile to every other recipient node of said first group, said first node establishing concurrent communication connections with non-failed nodes of said first group that have not received the particular subfile from said failed node, and said first node communicating the particular subfile to said non-failed nodes.

31. The system of claim 22 wherein said distribution technique further comprises:

said first group of recipient nodes attempting to communicate said file F to a second group comprising a plurality of recipient nodes.

32. The system of claim 31 wherein said distribution technique further comprises:

each recipient node of said first group attempting to communicate the subfile that it the recipient node received from said first node to every node of the second group.

33. The system of claim 32 wherein said detecting means comprises:

means for detecting a failed node in said first group when said nodes of said first group are attempting to communicate said file F to the second group.

34. The system of claim 32 wherein responsive to a recipient node of said first group being detected as failed, said distribution technique comprises:

each non-failed recipient node of said first group communicating the subfile that the non-failed recipient node received from said first node to every node of the second group; and

each non-failed recipient node of said first group communicating the subfile that the failed node received from said first node to a corresponding node of the second group.

35. The system of claim 34 wherein said distribution technique further comprises:

a recipient node of said first group communicating the subfile that the failed node received from said first node to a node of the second group corresponding to the failed node of the first group.

43. The system of claim 31 wherein said first group of recipient nodes attempt to communicate said file F to a second group comprising a plurality of recipient nodes.

44. The system of claim 43 wherein each recipient node of said first group to attempt to communicate the subfile that the recipient node received from said origin node to every node of the second group.

45. The system of claim 44 wherein said recipient nodes of said first group to further detect a failed node in said first group when said nodes of said first group are attempting to communicate said file F to the second group.

46. The system of claim 45 wherein responsive to a recipient node of said first group being detected as failed, each non-failed recipient node of said first group is operable to communicate the subfile that the non-failed recipient node received from said origin node to every node of the second group, and each non-failed recipient node of said first group to communicate the subfile that the failed node received from said origin node to a corresponding node of the second group.

47. The system of claim 46 wherein a recipient node of said first group to further communicate the subfile that the failed node received from said origin node to a node of the second group corresponding to the failed node of the first group.

49. The system of claim 22, wherein said plurality of recipient nodes of said first group attempting to exchange their respective subfiles received from said first node comprises:

said plurality of recipient nodes of said first group attempting to exchange their respective subfiles received from said first node such that each recipient node obtains all of said plurality of subfiles.

36. A system comprising:

an origin node comprising a processor to partition a file F into a plurality of subfiles, wherein said plurality of subfiles correspond in number to a number of recipient nodes in a first group to which said file is to be distributed;

said origin node to attempt to distribute all of said plurality of subfiles to said recipient nodes, wherein said origin node attempts to distribute a different one of said plurality of subfiles to each of said recipient nodes;

said recipient nodes to attempt to exchange their respective subfiles received from said origin node such that each recipient node obtains all of said plurality of subfiles;

said origin node to detect a failed node in said first group; and

said origin node to manage distribution of said file F upon detecting a failed node in said first group in a manner such that every non-failed node of said first group receives said file F.

37. The system of claim 36 wherein said attempting to distribute the plurality of subfiles from said origin node to a first group of recipient nodes comprises:

attempting to distribute the plurality of subfiles from said origin node to said plurality of recipient nodes of said first group concurrently.

38. The system of claim 36 wherein said plurality of recipient nodes of said first group attempting to exchange their respective subfiles further comprises:

each of said plurality of recipient nodes attempting to establishing concurrent communication connections to every other recipient node of said first group.

39. The system of claim 36 wherein said origin node to detect a recipient node of said first group as failed based at least in part on said origin node being unable to communicate a particular subfile to such recipient node.

40. The system of claim 36 wherein said attempting to distribute the plurality of subfiles from said origin node to said first group of recipient nodes comprises said origin node attempting to establish concurrent communication connections to the recipient nodes of said first group, and wherein said responsive to said origin node detecting a failed node in said first group such that said origin node is unable to communicate a particular subfile to such failed node, said origin node using the established concurrent communication connections with non-failed nodes of said first group to communicate the particular subfile to said non-failed nodes.

41. The system of claim 36 wherein said origin node to detect a failed node in said first group after said origin node has communicated a particular subfile to such failed node but before such failed node has communicated said particular subfile to every other recipient node of said first group.

42. The system of claim 41 wherein responsive to said origin node detecting a failed node in said first group after said origin node has communicated a particular subfile to such failed node but before such failed node has communicated said particular subfile to every other recipient node of said first group, said origin node establishing concurrent communication connections with non-failed nodes of said first group that have not received the particular subfile from said failed node, and said origin node communicating the particular subfile to said non-failed nodes.