Method for improving mean time to data loss (MTDL) in a fixed content distributed data storage

• US 9,305,011 B2
• Filed: 02/15/2007
• Issued: 04/05/2016
• Est. Priority Date: 07/27/2005
• Status: Active Grant

• Current Assignee: Hitachi Vantara, LLC (Hitachi, Ltd.)
• Original Assignee: Hitachi Data Systems Corporation (Hitachi, Ltd.)
• Inventors: Mason, Robert S. Jr.
• Primary Examiner(s): Smith, Marcus R
• Assistant Examiner(s): RAHGOZAR, OMEED DANIEL
• Application Number:
  US11/675,224
• Publication Number:
  US 20070189153A1
• Time in Patent Office: 3,337 Days
• Field of Search: 370/216, 711/114, 711/162, 707/202
• US Class Current: 1/1
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/181 providing write once read many [WORM] semantics
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself

Abstract

An archival storage cluster of preferably symmetric nodes includes a data protection management system that periodically organizes the then-available nodes into one or more protection sets, with each set comprising a set of n nodes, where “n” refers to a configurable “data protection level” (DPL). At the time of its creation, a given protection set is closed in the sense that each then available node is a member of one, and only one, protection set. When an object is to be stored within the archive, the data protection management system stores the object in a given node of a given protection set and then constrains the distribution of copies of that object to other nodes within the given protection set. As a consequence, all DPL copies of an object are all stored within the same protection set, and only that protection set. This scheme significantly improves MTDL for the cluster as a whole, as the data can only be lost if multiple failures occur within nodes of a given protection set. This is far more unlikely than failures occurring across any random distribution of nodes within the cluster.

Claims

1. A protection management method for a system of nodes, wherein each node comprises a processor, and a plurality of drives, the protection management method comprising:

grouping the plurality of nodes, for each of a plurality of data protection levels which define a number of nodes to store a copy of object data, into a group of one or more protection sets being associated with the respective data protection level, wherein a given node is a member of plural protection sets being associated with different data protection levels but only the member of one protection set among the one or more protection sets associated with the respective data protection level;

defining, for each of a plurality of protection levels, a plurality of protection sets being associated with respective protection levels;

storing copies of first object data that has a first protection level to each node in a first protection set which is associated with the first protection level based on a protection set map, and constraining distribution of the copies of the first object data to each node within the first protection set which is associated with the first protection level; and

storing copies of second object data that has a second protection level to each node in a second protection set which is associated with the second protection level based on the protection set map, and constraining distribution of the copies of the second object data to each node within the second protection set which is associated with the second protection level.

2. The protection management method as described in claim 1, wherein the protection level is defined as a data protection level.

3. The protection management method as described in claim 1, further comprising:

updating the protection set map by one of the nodes; and

broadcasting an updated protection set map to other nodes.

4. The protection management method as described in claim 3, wherein the step of updating the protection set map includes:

obtaining information about a current state of each of the nodes;

determining whether a given node in a first protection set is in a given state;

if the given node in the first protection set is in the given state, breaking the first protection set; and

forming a new protection set by grouping at least one other node that is then currently ungrouped with one or more nodes of the first protection set that has been broken.

5. The protection management method as described in claim 4, further including selecting the one of the nodes that performs the updating using leader election.

6. The protection management method as described in claim 1, further including periodically updating the protection set map upon a given occurrence.

7. The protection management method as described in claim 6, wherein the given occurrence is one of:

a failure of a given node within a protection set, and addition of a new node to the system.

8. The protection management method as described in claim 1, further comprising:

storing copies of third object data that has the first protection level to each node in a third protection set which is associated with the first protection level based on the protection set map, and constraining distribution of the copies of the third object data to each node within the third protection set which is associated with the first protection level.

9. A protection management method for a system including a plurality of nodes, the protection management method comprising:

grouping the plurality of nodes, for each of a plurality of data protection levels which define a number of nodes to store a copy of object data, into a group of one or more protection sets being associated with the respective data protection level, wherein a given node is a member of plural protection sets being associated with different data protection levels but only the member of one protection set among the one or more protection sets associated with the respective data protection level;

defining, for each of a plurality of protection levels, a plurality of protection sets being associated with respective protection levels, storing copies of first object data that has a first protection level to each node in a first protection set which is associated with the first protection level based on a protection set map, and constraining distribution of the copies of the first object data to each node within the first protection set which is associated with the first protection level; and

storing copies of second object data that has a second protection level to each node in a second protection set which is associated with the second protection level based on the protection set map, and constraining distribution of the copies of the second object data to each node within the second protection set which is associated with the second protection level.

10. The protection management method as described in claim 9, further including updating the protection set map.

11. The protection management method as described in claim 9, wherein the protection set map is updated periodically or upon a given occurrence.

12. The protection management method as described in claim 9, further comprising:

storing copies of third object data that has the first protection level to each node in a third protection set which is associated with the first protection level based on the protection set map and preventing distribution of the copies of the third object data to any other node in any other protection set which is associated with the first protection level.

13. A system, comprising:

a plurality of nodes, the plurality of nodes being grouped, for each of a plurality of data protection levels which define a number of nodes to be able to store a copy of object data, into a group of one or more protection sets being associated with the respective data protection level, wherein a given node is a member of plural protection sets being associated with different data protection levels but only the member of one protection set among the one or more protection sets associated with the respective data protection level; and

a network that couples the plurality of nodes;

wherein the system is configured to, upon receipt of first object data that has a first protection level, store copies of the first object data that has the first protection level to each node in a first protection set which is associated with the first protection level based on a protection set map, and constraining distribution of the copies of the first object data to each node within the first protection set which is associated with the first protection level, and

wherein the system is configured to, upon receipt of second object data that has a second protection level, store copies of the second object data that has the second protection level to each node in a second protection set which is associated with the second protection level based on the protection set map, and constraining distribution of the copies of the second object data to each node within the second protection set which is associated with the second protection level.

14. The system as described in claim 13, wherein one of the nodes is configured to update the protection set map periodically or upon a given occurrence.

15. The system as described in claim 14, wherein the one of the nodes that performs the update is further configured to broadcast the updated protection set map to other nodes.

16. The system as described in claim 14, wherein the protection set map is updated by the one of the nodes being configured to obtain information about a current state of each of the nodes, determine whether a given node in a third protection set is in a given state, break the first protection set when the given node in the third protection set is in the given state, and form a new protection set by grouping at least one other node that is then currently ungrouped with one or more nodes of the third protection set that has been broken.

17. The system as described in claim 13, wherein a number of nodes in each protection set of the given protection level is determined by the respective protection level.

18. The system as described in claim 13, wherein the nodes included in one of the protection sets for the respective protection level are different from the nodes included in another protection set for the respective protection level.

19. The system as described in claim 13, wherein the system is further configured to, upon receipt of third object data that has the first protection level, store copies of the third object data that has the first protection level to only each node in a third protection set which is associated with the first protection level based on the protection set map.