Resilient distributed storage system

• US 10,146,652 B2
• Filed: 02/11/2016
• Issued: 12/04/2018
• Est. Priority Date: 02/11/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hetzler, Steven R.
• Primary Examiner(s): Ko, Chae M
• Application Number:
  US15/041,999
• Publication Number:
  US 20170235631A1
• Time in Patent Office: 1,027 Days
• Field of Search: 714 624
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

One embodiment provides a method for storing data in a data storage system. The method comprises copying a data set into a buffer in a data storage center in response to receiving a write command for the data set; updating a content map; and sending to a second data storage center a parity write command for the data set. The method continues with copying the data set into a buffer in the second data storage center; updating a second content map; and sending to the first data storage center an acknowledgement indicating the parity write command is complete. Then in response to receiving the acknowledgement, updating the first content map; moving the data set to a storage unit in first data storage center; and acknowledging the data set is protected. Finally, releasing the data set from the first buffer; and updating the first map content.

Claims

1. A data storage system;

comprising;

a plurality of data storage centers, each comprising;

a set of storage units;

an encoder configured to encode local protection information and global protection information;

a parity map data structure configured to dynamically identify which data storage centers in the plurality of data storage centers participate in a given parity group;

a content map data structure configured to identify where among the plurality of data storage centers that data is stored and how to recover the data when a given data storage center is unreachable; and

a buffer configured to temporarily store data prior to loading into the set of storage units,

wherein the data is moved asynchronously between the plurality of data storage centers, parity for the plurality of data storage centers is maintained asynchronously by at least one parity data storage center, and each of the plurality of data storage centers is configured to issue a parity read command.

2. The system of claim 1, wherein a portion of available storage in a given data storage center of the plurality of data storage centers is configured as a parity data storage center.

3. The system of claim 2, each content map comprising:

a data set field indicating a data set being stored;

a source field indicating from which data storage center the data set originates;

a location field indicating whether the data set is in a buffer or in a storage unit; and

a state flag field indicating whether a local parity of the data set is complete, whether a global copy of the data set is complete and whether a global protection of the data set is complete.

4. The system of claim 3, each content map further comprising:

a recovery flag field indicating whether the data set is to be forwarded to the parity data center;

a recovery destination field indicating a location of the parity data storage center; and

a recovery source field indicating a given data storage center of the plurality of data storage centers for recovery.

5. The system of claim 4, each parity map comprising:

an identification of the first data storage center configured to store the data set;

an identification of the second data storage center configured to store the data set; and

an identification of the parity data storage center configured to maintain parity for the plurality of data storage centers.

6. The system of claim 5, wherein each data storage center is configured to:

in response to receiving a write command for the first data set;

copy the data set into a buffer in the given data storage center;

update a content map in the given data storage center;

send a first acknowledgement indicating the write command is complete; and

send to another second data storage center a parity write command for the first data set.

7. The system of claim 6, wherein each data storage center is further configured to:

in response to receiving an acknowledgement indicating the parity write command is complete;

update the content map in the given data storage center;

move the data set in the buffer to a storage unit in the given data storage center;

send a second acknowledgement indicating the data set is protected against loss; and

release the data set from the buffer and update the map content in the given data storage center.

8. The system of claim 1, wherein the local protection information is an erasure code with a minimum Hamming distance greater than 1, the parity read command includes a request source data center identifier, a requesting host identifier and a data set identifier, and the parity read command returns:

a data set flag indicating if the data set is parity or raw data, and a flag indicating that data has been returned to the requesting host.

9. A method for storing data in a data storage system;

comprising;

in response to receiving a write command for a first data set, copying the first data set into a first buffer in a first data storage center;

updating a first content map in the first data storage center;

sending a first acknowledgement indicating the write command is complete; and

sending to a second data storage center a parity write command for the first data set;

wherein the data storage system comprises a plurality of data storage centers, data is moved asynchronously between the plurality of data storage centers, parity for the plurality of data storage centers is maintained asynchronously by at least one parity data storage center, and each of the plurality of data storage centers is configured to issue a parity read command.

10. The method of claim 9, wherein:

each data storage center comprises;

a set of storage units;

an encoder configured to encode local protection information and global protection information;

a parity map;

a content map; and

a buffer configured to temporarily store data prior to loading into the set of storage units; and

a portion of available storage in a given data storage center of the plurality of data storage centers is configured as a parity data storage center.

11. The method of claim 10, further comprising:

copying the first data set into a second buffer in the second data storage center;

updating a second content map in the second data storage center; and

sending to the first data storage center an acknowledgement indicating the parity write command is complete.

12. The method of claim 11, further comprising:

in response to receiving the acknowledgement indicating the parity write command is complete, updating the first content map;

moving the first data set in the first buffer to a first storage unit in first data storage center; and

sending a second acknowledgement indicating the first data set is protected against loss.

13. The method of claim 12, further comprising:

releasing the first data set from the first buffer; and

updating the first content map.

14. The method of claim 13, each content map comprising:

a data set field indicating the data set being stored;

a source field indicating from which data storage center the data set originates;

a location field indicating whether the data set is in a buffer or in a storage unit; and

a state flag field indicating whether a local parity of the data set is complete, whether a global copy of the data set is complete and whether a global protection of the data set is complete.

15. The method of claim 14, each content map further comprising:

a recovery flag field indicating whether the data set is to be forwarded to the parity data storage center;

a recovery destination field indicating a location of the parity data storage center; and

a recovery source field indicating a given data storage center of the plurality of data storage centers for recovery.

16. The method of claim 13, each parity map comprising:

an identification of the first data storage center configured to store the data set;

an identification of the second data storage center configured to store the data set; and

an identification of the parity data storage center configured to maintain parity for the plurality of data storage centers.

17. A system comprising a computer processor, a computer-readable hardware storage device, and program code embodied with the computer-readable hardware storage device for execution by the computer processor to implement a method comprising:

in response to receiving a write command for a first data set, copying the first data set into a first buffer in a first data storage center;

updating a first content map in the first data storage center;

sending a first acknowledgement indicating the write command is complete;

sending to a second data storage center a parity write command for the first data set;

copying the first data set into a second buffer in the second data storage center;

updating a second content map in the second data storage center;

sending to the first data storage center an acknowledgement indicating the parity write command is complete;

in response to receiving the acknowledgement indicating the parity write command is complete, updating the first content map;

moving the first data set in the first buffer to a first storage unit in first data storage center;

sending a second acknowledgement indicating the first data set is protected against loss;

releasing the first data set from the first butler; and

updating the first content map,

wherein the system comprises a plurality of data storage centers, data is moved asynchronously between the plurality of data storage centers, a portion of available storage in a given data storage center of the plurality of data storage centers is configured as a parity data storage center, parity for the plurality of data storage centers is maintained asynchronously by at least one parity data storage center, and each of the plurality of data storage centers is configured to issue a parity read command.

18. The system of claim 17, each content map comprising:

a data set field indicating the data set being stored;

a source field indicating from which data storage center the data set originates;

a location field indicating whether the data set is in a buffer or in a storage unit;

a state flag field indicating whether a local parity of the data set is complete, whether a global copy of the data set is complete and whether a global protection of the data set is complete;

a recovery flag field indicating whether the data set is to be forwarded to the parity data storage center;

a recovery destination field indicating a location of the parity data storage center; and

a recovery source field indicating a given data storage center of the plurality of data storage centers for recovery.

19. The system of claim 18, each parity map comprising:

an identification of the first data storage center configured to store the data set;

an identification of the second data storage center configured to store the data set; and

an identification of the parity data storage center configured to maintain parity for the plurality of data storage centers.