Data replication facility for distributed computing environments

• US 7,054,910 B1
• Filed: 12/20/2001
• Issued: 05/30/2006
• Est. Priority Date: 12/20/2001
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Krishnan, Sriram, Bauer, Andreas L., Lazar, Gregory W., Nordin, Richard J.
• Primary Examiner(s): Lim, Krisna
• Application Number:
  US10/027,694
• Time in Patent Office: 1,622 Days
• Field of Search: 709208-211, 707200-204, 714/6, 714/42, 714/15, 714/48
• US Class Current: 709/208
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: H04L 61/4552 Lookup mechanisms between a plurality of directories; Synchronisation of directories, e.g. metadirectories
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A data replication facility for distributed computing environments. A computer network having a plurality of network nodes utilizes a distributed directory provider service (DPS) having an established master node. The DPS supports a file replication service (FRS). The FRS establishes one of the nodes as originator node which receives new or updated files from one or more user/GUIs and/or from one or more software providers such as a security provider. The originator node in cooperation with the master node establish a backup copy of the new or updated file in the master node. Thereafter, the originator node publishes a File Version Variable (FVV) representation of the new or updated file to other network nodes (slave nodes) which obtain such file from the originator or, if need be, from the backup master node. Object observers are utilized to determine changes to the file version variables thereby triggering the downloading of new or updated files into the network nodes, whereby data file replication is accomplished throughout the network. In addition to avoiding a single point of failure, embodiments of the present invention also are network-topology independent. Additional syncing threads are employed as part of the file replication service to further ensure synchronization of the network nodes'"'"' data files within a predetermined interval, regardless of network failure modes. Embodiments of the present invention are particularly useful with networks of the client-server storage network variety.

Claims

1. In a computer network having a plurality of nodes, a file replication system comprising:

means, distributed on each of said nodes, capable of receiving an updated file in any one of said nodes;

means, distributed throughout said plurality of nodes and responsive to receiving said file in a certain one of said nodes, for replicating said updated file in all other of said nodes in a manner that is network-topology independent and avoids a single point of failure;

wherein said certain one of said nodes is the originator node and said replicating means further comprises;

means for establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

means for storing said updated file on said master node as a backup file; and

,

in each of said slave nodes, means for updating a particular file corresponding to said updated file;

means for associating a file version variable with said updated file;

said storing means comprising;

local workspace means for receiving said updated file in said master node;

means for error checking said file version variable to confirm validity of said file version variable;

global workspace means adapted to receive said updated file from said local workspace means; and

,

means, responsive to operation of said error checking means confirming said validity, for transferring said updated file to said global workspace means; and

,

means, responsive to operation of said transferring means, for communicating both creation of said backup file to said originator node and availability of said backup file to said slave nodes.

2. The system of claim 1 and further comprising:

means, included within said master node, for communicating both creation of said backup file to said originator node and availability of said backup file to said slave nodes.

3. The system of claim 2 and further comprising:

said originator node including means, responsive to operation of said communicating means, for publishing a representation of said updated file to said slave nodes; and

means, within each of said slave nodes and responsive to operation of said publishing means, for commanding its respective node to obtain said updated file from said originator node.

4. The system of claim 2 and wherein operation of said communicating means communicating said creation of said backup file to said originator node comprises a success note.

5. The system of claim 3 and further comprising:

said originator node including means for establishing a updated file version variable as said representation of said file; and

said publishing means includes means for publishing said updated file version variable to said slave nodes.

6. The system of claim 5 and further comprising:

said replicating means including means for establishing a particular file version variable corresponding to said particular file;

object observer means, in said each of said slave nodes, for observing change from said particular file version variable to said updated file version variable; and

,

said commanding means including means, responsive to operation of said object observer means, for downloading said updated file from said originator node into said particular file in said each of said slave nodes.

7. The system of claim 3 further comprising:

means, responsive to operation of said commanding means not obtaining said updated file from said originator node, for further commanding said respective node to obtain said backup file from said master node.

8. The system of claim 1 and wherein:

said local workspace means includes other means for receiving said updated file in said originator node; and

,

said global workspace means includes other means for receiving said updated file from said other means in preparation to download said updated file to any of said slave nodes upon request from said any of said slave nodes.

9. The system of claim 8 and wherein said replicating means further comprises:

at least one data file including a data word with its corresponding file version variable.

10. The system of claim 8 and wherein said local workspace means further includes:

multiple source means for receiving said updated file from a new-data-supplier group consisting of a network user, a security provider, and other providers.

11. The system of claim 10 and wherein said multiple source receiving means includes additional means for receiving additional updated files from said new-data-supplier group.

12. The system of claim 1 and wherein said receiving means is further capable of receiving multiple updated files, each of said files being received from a different network user.

13. The system of claim 1 further comprising:

means, responsive to operation of said error checking means not confirming said validity, for flagging an error, stopping operation of said file replicating means on said updated file, and preparing said file replication system to receive a next successive updated file.

14. In a computer network having a plurality of nodes, a file replication system comprising:

means, distributed on each of said nodes, capable of receiving an updated file in any one of said nodes;

means, distributed throughout said plurality of nodes and responsive to receiving said file in a certain one of said nodes, for replicating said updated file in all other of said nodes in a manner that is network-topology independent and avoids a single point of failure;

wherein said certain one of said nodes is the originator node and said replicating means further comprises;

means for establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

means for storing said updated file on said master node as a backup file;

in each of said slave nodes, means for updating a particular file corresponding to said updated file; and

means for establishing a particular file version variable corresponding to said particular file;

polling means for allowing said each of said slave nodes to periodically poll said master node to determine if said particular file contents matches said updated file contents; and

,

synchronizing means, responsive to operation of said polling means determining that said particular file contents do not match said updated file contents, for synchronizing said particular file contents with said updated file contents.

15. The system of claim 14 further comprising:

means for establishing a DPS version number to identify the current version of DPS in said network;

means for establishing a DDB version number to identify the current version of DDB in said network;

first means for comparing said DPS version number on said each of said slave nodes with said DPS version number on said master node to obtain a respective DPS version number match;

said synchronizing means,

responsive to operation of said first comparing means obtaining said terminating further operation of said synchronizing means with respect to the current said poll on said each of said certain of said slave nodes, and

responsive to operation of said first comparing means not obtaining said respective DPS version number match on each of the remainder of said slave nodes for achieving said DDB version number match on said each of the remainder of said slave nodes;

second means, responsive to operation of said first comparing means not obtaining said respective DPS version number match, for comparing said particular file version variable on said each of the remainder of said slave nodes with said updated file version variable on said master node to obtain a respective file version variable match; and

,

said synchronizing means,

responsive to operation of said second comparing means obtaining a file version variable match on each of a portion of said remainder of said slave nodes for terminating operation of said synchronizing means with respect to the current said poll on said each of said portion, and

responsive to operation of said second means not obtaining a file version variable match on each of the remaining portion of said remainder of said slave nodes for achieving said file version variable match on said each of the remaining portion;

whereby said particular file contents matches said updated file contents in said each of said slave nodes.

16. In a computer network having a plurality of nodes, a file replication system comprising:

means, distributed on each of said nodes, capable of receiving an updated file in any one of said nodes;

means, distributed throughout said plurality of nodes and responsive to receiving said file in a certain one of said nodes, for replicating said updated file in all other of said nodes in a manner that is network-topology independent and avoids a single point of failure;

wherein said certain one of said nodes is the originator node and said replicating means further comprises;

means for establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

means for storing said updated file on said master node as a backup file;

in each of said slave nodes, means for updating a particular file corresponding to said updated file; and

means for establishing a particular file version variable corresponding to said particular file, and wherein each of said plurality of nodes is a storage system having storage media on which both said particular file version variable and said particular file contents are stored;

polling means, for comparing said particular file version variable stored on said storage media in said each of said nodes with said particular file version variable stored elsewhere in its respective node to determine a particular file version variable match for said each of said nodes; and

synchronizing means, responsive to operation of said polling means determining said particular file version variable match was not achieved for certain of said nodes for achieving said particular file version variable match for each of said certain nodes.

17. The system of claim 16 and wherein said storage media is at least one storage disk.

18. A computer program product for use in a computer network having a plurality of nodes, said computer program product including a tangible computer usable medium having computer readable program code thereon for file replication, said program code comprising:

program code, distributed on each of said nodes, capable of receiving an updated file in any one of said nodes;

program code, distributed throughout said plurality of nodes and responsive to receiving said file in a certain one of said nodes, for replicating said updated file in all other of said nodes in a manner that is network-topology independent;

wherein said certain one of said nodes is the originator node and said replicating program code further comprises;

program code for establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

program code for storing said updated file on said master node as a backup file; and

,

in each of said slave nodes, program code for updating a particular file corresponding to said updated file;

program code for associating a file version variable with said updated file;

said storing program code comprising;

local workspace program code for receiving said updated file in said master node;

program code for error checking said file version variable to confirm validity of said file version variable;

global workspace program code adapted to receive said updated file from said local workspace program code; and

,

program code, responsive to operation of said error checking program code confirming said validity for transferring said updated file to said global workspace program code; and

,

program code, responsive to operation of said transferring program code, for communicating both creation of said backup file to said originator node and availability of said backup file to said slave nodes.

19. The computer program product of claim 18 and wherein said replicating program code includes:

additional replication program code for replicating said updated file in all other of said nodes in a manner that avoids a single point of failure.

20. The computer program product of claim 18 and further comprising:

program code, included within said master node, for communicating both creation of said backup file to said originator node and availability of said backup file to said slave nodes.

21. The computer program product of claim 20 and further comprising:

said originator node including program code, responsive to operation of said communicating program code, for publishing a representation of said updated file to said slave nodes; and

program code, within each of said slave nodes and responsive to operation of said publishing program code, for commanding its respective node to obtain said updated file from said originator node.

22. The computer program product of claim 20 and wherein operation of said communicating program code communicating said creation of said backup file to said originator node comprises a success note.

23. The computer program product of claim 21 and further comprising:

said originator node including program code for establishing an updated file version variable as said representation of said updated file; and

said publishing program code includes program code for publishing said updated file version variable to said slave nodes.

24. The computer program product of claim 23 and further comprising:

said replicating program code including program code for establishing a particular file version variable corresponding to said particular file;

object observer program code, in said each of said slave nodes, for observing change from said particular file version variable to said updated file version variable; and

,

said commanding program code including program code, responsive to operation of said object observer program code, for downloading said updated file from said originator node into said particular file in said each of said slave nodes.

25. The computer program product of claim 21 further comprising:

program code, responsive to operation of said commanding program code not obtaining said updated file from said originator node, for further commanding said respective node to obtain said backup file from said master node.

26. The computer program product of claim 18 and wherein said receiving program code is further capable of receiving multiple updated files, each of said files being received from a different network user.

27. The computer program product of claim 18 further comprising:

program code, responsive to operation of said error checking program code not confirming said validity, for flagging an error, stopping operation of said file replicating program code on said updated file, and preparing said file replication computer program product to receive a next successive updated file.

28. The computer program product of claim 18 and wherein said replicating program code comprises:

local workspace program code for receiving said updated file in said originator node; and

,

global workspace program code, operatively coupled to said local workspace program code, for receiving said updated file from said local workspace program code in preparation to download said updated file to any of said slave nodes upon request from said any of said slave nodes.

29. The computer program product of claim 28 and wherein said replicating program code further comprises:

at least one data file including a data word with its corresponding file version variable.

30. The computer program product of claim 28 and wherein said local workspace program code further includes:

multiple source program code for receiving said updated file from a new-data-supplier group consisting of a network user, a security provider, and other providers.

31. The computer program product of claim 30 and wherein said multiple source receiving program code includes additional program code for receiving additional updated files from said new-data-supplier group.

32. A computer program product for use in a computer network having a plurality of nodes, said computer program product including a tangible computer usable medium having computer readable program code thereon for file replication, said program code comprising:

program code, distributed on each of said nodes, capable of receiving an updated file in any one of said nodes;

program code, distributed throughout said plurality of nodes and responsive to receiving said file in a certain one of said nodes, for replicating said updated file in all other of said nodes in a manner that is network-topology independent;

wherein said certain one of said nodes is the originator node and said replicating program code further comprises;

program code for establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

program code for storing said updated file on said master node as a backup file; and

,

in each of said slave nodes, program code for updating a particular file corresponding to said updated file; and

program code for establishing a particular file version variable corresponding to said particular file;

polling program code for allowing said each of said slave nodes to periodically poll said master node to determine if said particular file contents matches said updated file contents; and

,

synchronizing program code, responsive to operation of said polling program code determining that said particular file contents do not match said updated file contents, for synchronizing said particular file contents with said updated file contents.

33. The computer program product of claim 32 further comprising:

program code for establishing a DPS version number to identify the current version of DPS in said network;

program code for establishing a DDB version number to identify the current version of DDB in said network;

first program code for comparing said DPS version number on said each of said slave nodes with said DPS version number on said master node to obtain a respective DPS version number match;

said synchronizing program code,

responsive to operation of said first comparing program code obtaining said respective DPS version number match on each of certain of said slave nodes for terminating further operation of said synchronizing program code with respect to the current said poll on said each of said certain of said slave nodes, and

responsive to operation of said first comparing program code not obtaining said respective DPS version number match on each of the remainder of said slave nodes for achieving said DDB version number match on said each of the remainder of said slave nodes;

second program code, responsive to operation of said first comparing program code not obtaining said respective DPS version number match, for comparing said particular file version variable on said each of the remainder of said slave nodes with said updated file version variable on said master node to obtain a respective file version variable match; and

,

said synchronizing program code,

responsive to operation of said second comparing program code obtaining a file version variable match on each of a portion of said remainder of said slave nodes for terminating operation of said synchronizing program code with respect to the current said poll on said each of said portion, and

responsive to operation of said second program code not obtaining a file version variable match on each of the remaining portion of said remainder of said slave nodes for achieving said file version variable match on said each of the remaining portion;

whereby said particular file contents matches said updated file contents in said each of said slave nodes.

34. In a computer network having a plurality of nodes, a file replication method executed by a computer, said method comprising:

arranging for receipt of an updated file in any one of said nodes;

receiving said updated file in a certain one of said nodes;

replicating said file in all other of said nodes in a manner that is network-topology independent;

wherein said certain one of said nodes is the originator node and said replicating further comprises;

establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

storing said updated file on said master node as a backup file; and

,

in each of said slave nodes, updating a particular file corresponding to said updated file;

associating a file version variable with said updated file;

said storing comprising;

receiving said updated file in local workspace of said master node;

error checking said file version variable to confirm validity of said file version variable;

adapting global workspace to receive said updated file from said local workspace; and

,

transferring said updated file to said global workspace responsive to said error checking confirming said validity; and

,

communicating both creation of said backup file to said originator node and availability of said backup file to said slave nodes responsive to said transferring.

35. The method of claim 34 and wherein said replicating program code includes:

replicating said updated file in all other of said nodes in a manner that avoids a single point of failure.

36. The method of claim 34 further comprising:

communicating both creation of said backup file to said originator node and availability of said backup file to said slave nodes.

37. The method of claim 36 further comprising:

publishing a representation of said updated file to said slave nodes; and

within each of said slave nodes, commanding its respective node to obtain said updated file from said originator node.

38. The method of claim 36 and wherein communicating said creation of said backup file to said originator node comprises a success note.

39. The method of claim 37 further comprising:

establishing an updated file version variable as said representation of said updated file; and

publishing said updated file version variable to said slave nodes.

40. The method of claim 39 further comprising:

establishing a particular file version variable corresponding to said particular file;

observing change from said particular file version variable to said updated file version variable; and

,

downloading said updated file from said originator node into said particular file in said each of said slave nodes.

41. The method of claim 37 further comprising:

further commanding said respective node to obtain said backup file from said master node responsive to said commanding not obtaining said updated file from said originator node.

42. The method of claim 34 further comprising:

receiving said updated file in local workspace of said originator node; and

,

receiving said updated file into global workspace from said local workspace in preparation to download said updated file to any of said slave nodes upon request from said any of said slave nodes.

43. The method of claim 42 and wherein said replicating further comprises:

including a data word with its corresponding file version variable in at least one data file.

44. The method of claim 42 further comprising:

receiving said updated file from a new-data-supplier group consisting of a network user, a security provider, and other providers.

45. The method of claim 44 further comprising:

receiving additional updated files from said new-data-supplier group.

50. The method of claim 34 responsive to said error checking not confirming said validity, said method further comprising:

flagging an error;

stopping said file replicating on said updated file; and

,

preparing said file replication method to receive a next successive updated file.

51. The method of claim 34 further comprising:

receiving multiple updated files, each of said files being received from a different network user.

46. In a computer network having a plurality of nodes, a file replication method executed by a computer, said method comprising:

arranging for receipt of an updated file in any one of said nodes;

receiving said updated file in a certain one of said nodes;

replicating said file in all other of said nodes in a manner that is network-topology independent;

wherein said certain one of said nodes is the originator node and said replicating further comprises;

establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

storing said updated file on said master node as a backup file; and

,

in each of said slave nodes, updating a particular file corresponding to said updated file;

establishing a particular file version variable corresponding to said particular file;

said each of said slave nodes periodically polling said master node to determine if said particular file contents matches said updated file contents;

determining that said particular file contents do not match said updated file contents; and

,

synchronizing said particular file contents with said updated file contents.

47. The method of claim 46 further comprising:

establishing a DPS version number to identify the current version of DPS in said network;

establishing a DDB version number to identify the current version of DDB in said network;

first comparing said DPS version number on said each of said slave nodes with said DPS version number on said master node to obtain a respective DPS version number match;

said synchronizing,

responsive to said first comparing obtaining said respective DPS version number match on each of certain of said slave nodes, terminating further operation of said synchronizing with respect to the current poll on said each of said certain of said slave nodes, and

further responsive to said first comparing not obtaining said respective DPS version number match on each of the remainder of said slave nodes, achieving said DDB version number match on said each of the remainder of said slave nodes;

second comparing said particular file version variable on said each of the remainder of said slave nodes with said updated file version variable on said master node obtaining said respective DPS version number match; and

,

said synchronizing,

further responsive to said second comparing obtaining a file version variable match on each of a portion of said remainder of said slave nodes, terminating operation of said synchronizing with respect to said current poll on said each of said portion, and

further responsive to said second comparing not obtaining a file version variable match on each of the remaining portion of said remainder of said slave nodes, achieving said file version variable match on said each of the remaining portion;

whereby said particular file contents matches said updated file contents in said each of said slave nodes.

48. In a computer network having a plurality of nodes, a file replication method executed by a computer, said method comprising:

arranging for receipt of an updated file in any one of said nodes;

receiving said updated file in a certain one of said nodes;

replicating said file in all other of said nodes in a manner that is network-topology independent;

wherein said certain one of said nodes is the originator node and said replicating further comprises;

establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

storing said updated file on said master node as a backup file; and

,

in each of said slave nodes, updating a particular file corresponding to said updated file; and

wherein each of said plurality of nodes is a storage system having storage media and wherein said replicating further comprises;

establishing a particular file version variable corresponding to said particular file;

storing said particular file version variable and said particular file contents on said storage media to obtain stored particular file version variable and contents;

comparing said particular file version variable stored on said storage media in said each of said nodes with said particular file version variable stored elsewhere in its respective node to determine a particular file version variable match for said each of said nodes; and

,

synchronizing, responsive to comparing determining said particular file version variable match was not achieved for certain of said nodes, achieving said particular file version variable match for each of said certain nodes.

49. The method of claim 48 and wherein said storage media is at least one storage disk.

52. A method for synchronizing data in a network having a master node and at least one slave node comprising:

establishing a DPS in said master node and said at least one slave node;

for each said slave node, periodically polling said master node to determine if the version number of said DPS in said master node matches the version number of said DPS in said at least one slave node;

for each said slave node, terminating said method if said master node DPS version number matches said at least one slave node DPS version number; and

,

if said master node DPS version number does not match said at least one slave node DPS version number thereby providing a mismatch;

first determining if said mismatch is due to a DDB version number mismatch only and, if so, first replicating IP addresses of said at least one slave node and terminating said method; and

,

if said mismatch is not due to a DDB version number mismatch only, second determining if said mismatch is due to a FVV mismatch only and, if so, second replicating new files associated with a new said FVV and terminating said method; and

,

if said mismatch is due to both said DDB version number mismatch and said FVV mismatch, performing said first replicating and said replicating and terminating said method.

53. A method for synchronizing data on storage media in a network having a master node and at least one other node including slave node and originator node, said method comprising:

establishing a DPS in said master node and said at least one other node;

for each node in said network, periodically comparing each FVV stored in its respective said storage media with its corresponding FVV stored in its DPS;

if there is a match between said storage media FVV and its corresponding said DPS FVV, terminating said method;

if there is no match between said storage media FVV and its corresponding said DPS FVV thereby providing a first mismatch, first determining if said first mismatch occurred on said master node or on said at least one other node;

if said first mismatch occurred on said at least one other node second determining if said first mismatch is due to a missing file or an extra file on said other node storage media; and

if due to said missing file, retrieving said missing file from said originator node or said master node and terminating said method; and

,

if due to said extra file, ignoring said extra file and deleting said extra file from said media and terminating said method; and

,

if said first mismatch occurred on said master node, third determining if said mismatch is due to a missing file or an extra file on said master node storage media; and

if due to said missing file, removing the corresponding file from said DPS in said master node and terminating said method; and

,

if due to said extra file, adding said extra file to said DPS in said master node and terminating said method.

54. A computer program product for use in a computer network having a plurality of nodes, said computer program product including a tangible computer usable medium having computer readable program code thereon for file replication, said program code comprising:

program code, distributed on each of said nodes, capable of receiving an updated file in any one of said nodes;

program code, distributed throughout said plurality of nodes and responsive to receiving said file in a certain one of said nodes, for replicating said updated file in all other of said nodes in a manner that is network-topology independent;

wherein said certain one of said nodes is the originator node and said replicating program code further comprises;

program code for establishing another of said nodes as master node, said plurality of nodes except for both said originator node and said master node being slave nodes;

program code for storing said updated file on said master node as a backup file; and

,

in each of said slave nodes, program code for updating a particular file corresponding to said updated file; and

program code for establishing a particular file version variable corresponding to said particular file, and wherein each of said plurality of nodes is a storage computer program product having storage media on which both said particular file version variable and said particular file contents are stored;

polling program code, for comparing said particular file version variable stored on said storage media in said each of said nodes with said particular file version variable stored elsewhere in its respective node to determine a particular file version variable match for said each of said nodes; and

synchronizing program code, responsive to operation of said polling program code determining said particular file version variable match was not achieved for certain of said nodes for achieving said particular file version variable match for each of said certain nodes.

55. The computer program product of claim 54 and wherein said storage media is at least one storage disk.