Distributed storage system

• US 10,496,479 B2
• Filed: 08/22/2018
• Issued: 12/03/2019
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Akutsu, Hiroaki, Kawamura, Shunji, Yasunaga, Kota, Yamamoto, Takahiro, Kawamura, Atsushi
• Primary Examiner(s): Torres, Joseph D
• Application Number:
  US16/108,265
• Publication Number:
  US 20180357127A1
• Time in Patent Office: 468 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/2906 using block codes H03M13/2957 takes precedence
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A first node group including at least three nodes is predefined in a distributed storage system. Each node of the first node group is configured to send data blocks stored in storage devices managed by the node to other nodes belonging to the first node group. A first node is configured to receive data blocks from two or more other nodes in the first node group. The first node is configured to create a redundant code using a combination of data blocks received from the two or more other nodes and store the created redundant code to a storage device different from storage devices holding the data blocks used to create the redundant code. Combinations of data blocks used to create at least two redundant codes in redundant codes created by the first node are different in combination of logical addresses of constituent data blocks.

Claims

1. A distributed storage system comprising:

a plurality of nodes configured to communicate with one another via a network, each one of the nodes comprises a respective volume and is configured to manage its respective volume,

wherein each of the plurality of nodes is further configured to;

store user data and secondary redundant data in the respective volume thereof, and

create the secondary redundant data stored in the respective volume thereof based on blocks of data of other nodes different from one another, the blocks of data being user data stored in the other nodes and primary redundant data created from the user data stored in the other nodes,

wherein when restoring the user data, each of the plurality of nodes is configured to;

restore a first primary redundant data based on a first user data stored in one of the nodes;

restore a plurality of second primary redundant data based on the restored first primary redundant data and secondary redundant data stored in one of the nodes; and

restore a second user data based on the plurality of second primary redundant data.

2. The distributed storage system according to claim 1, wherein each of the plurality of nodes is further configured to create the secondary redundant data based on blocks of the user data and create the secondary redundant data based on blocks of the primary redundant data.

3. The distributed storage system according to claim 1, wherein, in each of the plurality of nodes, the secondary redundant data stored in the respective volume thereof is independent of the user data stored in the respective volume thereof.

4. The distributed storage system according to claim 1, wherein the plurality of nodes include a first node, a second node and a third node, and

wherein the first user data is stored in the first node, the first secondary redundant data is stored in the second node different from the first node, and the second data is the primary redundant data created based on the user data stored in the third node different from the first node and the second node.

5. The distributed storage system according to claim 1, wherein the plurality of nodes include a first node, a second node, a third node and a fourth node,

wherein the first node stores received first user data in the respective volume thereof and stores the secondary redundant data created from blocks of the user data stored in the second node, the third node and the fourth node,

wherein the second node stores received second user data in the respective volume thereof and stores the secondary redundant data created from one of the blocks of the user data stored in the first node and the blocks of the primary redundant data created from the user data of the third node and the fourth node,

wherein the third node stores received third user data in the respective volume thereof and stores the secondary redundant data created from the blocks of the primary redundant data created from the user data of the first node, the second node and the fourth node,

wherein the fourth node stores received fourth user data in the respective volume thereof and stores the secondary redundant data created from the blocks of the primary redundant data created from the user data of the first node, the second node and the third node.

6. The distributed storage system according to claim 5, wherein the secondary redundant data stored by the third node and the secondary redundant data stored by the fourth node are created from different blocks of the primary redundant data.

7. The distributed storage system according to claim 1, wherein the primary redundant data is not stored in the node and each of the plurality of nodes is configured to generate the primary redundant data when storing the user data and restoring the user data.

8. A method of storing and restoring data in a distributed storage system which includes a plurality of nodes configured to communicate with one another via a network, each of the plurality of nodes comprises a respective volume and is configured to manage its respective volume, the method comprising:

storing, by each one of the nodes, user data in the respective volume thereof;

creating, by each one of the nodes, secondary redundant data based on blocks of data of other nodes different from one another, the blocks of data being user data stored in the other nodes and primary redundant data created from the user data stored in the other nodes; and

storing, by each one of the nodes, the respectively created secondary redundant data in the respective volume thereof;

wherein when restoring the user data, the method comprising;

restoring a first primary redundant data based on a first user data stored in one of the nodes;

restoring a plurality of second primary redundant data based on the restored first primary redundant data and secondary redundant data stored in one of the nodes; and

restoring a second user data based on the plurality of second primary redundant data.

9. The method according to claim 8, wherein each of the plurality of nodes is further configured to create the secondary redundant data based on blocks of the user data and create the secondary redundant data based on blocks of the primary redundant data.

10. The method according to claim 8, wherein, in each of the plurality of nodes, the secondary redundant data stored in the respective volume thereof is independent of the user data stored in the respective volume thereof.

11. The method according to claim 8, wherein the plurality of nodes include a first node, a second node and a third node, and

wherein the first user data is stored in the first node, the first secondary redundant data is stored in the second node different from the first node, and the second data is the primary redundant data created based on the user data stored in the third node different from the first node and the second node.

12. The method according to claim 8, wherein the plurality of nodes include a first node, a second node, a third node and a fourth node,

wherein the first node stores received first user data in the respective volume thereof and stores the secondary redundant data created from blocks of the user data stored in the second node, the third node and the fourth node,

wherein the second node stores received second user data in the respective volume thereof and stores the secondary redundant data created from one of the blocks of the user data stored in the first node and the blocks of the primary redundant data created from the user data of the third node and the fourth node,

wherein the third node stores received third user data in the respective volume thereof and stores the secondary redundant data created from the blocks of the primary redundant data created from the user data of the first node, the second node and the fourth node, and

wherein the fourth node stores received fourth user data in the respective volume thereof and stores the secondary redundant data created from the blocks of the primary redundant data created from the user data of the first node, the second node and the third node.

13. The method according to claim 12, wherein the secondary redundant data stored by the third node and the secondary redundant data stored by the fourth node are created from different blocks of the primary redundant data.

14. The method according to claim 8, wherein the primary redundant data is not stored in the node and each of the plurality of nodes is configured to generate the primary redundant data when storing the user data and restoring the user data.