Distributed data storage

• US 9,495,432 B2
• Filed: 01/31/2014
• Issued: 11/15/2016
• Est. Priority Date: 10/24/2008
• Status: Active Grant

• Current Assignee: Compuverde AB (Pure Storage, Inc.)
• Original Assignee: Compuverde AB (Pure Storage, Inc.)
• Inventors: Bernbo, Stefan, Melander, Christian, Persson, Roger, Petersson, Gustav
• Primary Examiner(s): Chbouki, Tarek
• Application Number:
  US14/169,699
• Publication Number:
  US 20140149351A1
• Time in Patent Office: 1,019 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

The present invention relates to a distributed data storage system comprising a plurality of storage nodes. Using unicast and multicast transmission, a server application may read and write data in the storage system. Each storage node may monitor reading and writing operations on the system as well as the status of other storage nodes. In this way, the storage nodes may detect a need for replication of files on the system, and may carry out a replication process that serves to maintain a storage of a sufficient number of copies of files with correct versions at geographically diverse locations.

Claims

1. A method for writing data to a data storage system, the method being employed in a server that accesses data in the data storage system via a communication network, the method comprising:

the server sending a multicast storage query to a plurality of data storage nodes of the data storage system, the multicast storage query indicating the server is requesting identification of data storage nodes that can store a data file;

the server receiving a plurality of responses from a subset of the plurality of data storage nodes, each response comprising geographic data relating to a geographic position of the data storage node that sent the response;

the server selecting at least two data storage nodes in the subset based on the geographic data included in the responses, wherein the selection ensures that there is at least a requisite level of geographical diversity between the at least two data storage nodes;

the server sending a data file and a data identifier that corresponds to the data file to the at least two selected data storage nodes, wherein the server sends a host list for the data file that indicates which data storage nodes are being configured to store the data file;

the server receiving acknowledgement information from one or more of the at least two selected data storage nodes, wherein the server determines from the acknowledgement information that a first data storage node of the at least two selected data storage nodes successfully stored the data file and that at least a second data storage node of the at least two selected data storage nodes did not successfully store the data file; and

the server sending an indication that a data replication procedure should be initiated based on determining that at least the second data storage node of the at least two selected data storage nodes did not successfully store the data file, wherein the indication is sent to at least the first data storage node in order to trigger at least the first data storage node that did successfully store the data file to perform the data replication procedure for the at least one file within the data storage system such that the data file is replicated to at least a third data storage node in the data storage system.

2. A method according to claim 1, wherein the geographic data is indicative of a latitude and longitude of the data storage node that sent the response.

3. A method according to claim 2, wherein each response further indicates a system age for the data storage node that sent the response.

4. A method according to claim 2, wherein each response further indicates a system load for the data storage node that sent the response.

5. A method according to claim 2, wherein the multicast storage query includes the data identifier for the data file.

6. A method according to claim 2, wherein at least three nodes are selected.

7. A method according to claim 1, wherein data storage nodes in a storage group are identified in a node list that is separate from the host list, the node list indicating Internet Protocol (IP) addresses of the data storage nodes in the node group.

8. A server adapted for writing data to a data storage system comprising a plurality of data storage nodes, the server comprising a computer readable data storage medium comprising instructions that when executed result in at least:

the server sending a multicast storage query to a plurality of data storage nodes of the data storage system, the multicast storage query indicating the server is requesting identification of data storage nodes that can store a data file;

the server receiving a plurality of responses from a subset of the plurality of data storage nodes, each response comprising geographic data relating to a geographic position of the data storage node that sent the response;

the server selecting at least two data storage nodes in the subset based on the geographic data included in the responses, wherein the selection ensures that there is at least a requisite level of geographical diversity between the at least two data storage nodes;

the server sending a data file and a data identifier that corresponds to the data file to the at least two selected data storage nodes, wherein the server sends a host list for the data file that indicates which data storage nodes are being configured to store the data file;

the server receiving acknowledgement information from one or more of the at least two selected data storage nodes, wherein the server determines from the acknowledgement information that a first data storage node of the at least two selected data storage nodes successfully stored the data file and that at least a second data storage node of the at least two selected data storage nodes did not successfully store the data file;

the server sending an indication that a data replication procedure should be initiated based on determining that at least the second data storage node of the at least two selected data storage nodes did not successfully store the data file, wherein the indication is sent to at least the first data storage node in order to trigger at least the first data storage node that did successfully store the data file to perform the data replication procedure for the at least one file within the data storage system such that the data file is replicated to at least a third data storage node in the data storage system.

9. A server according to claim 8, wherein the geographic data is indicative of a latitude and longitude of the data storage node that sent the response.

10. A server according to claim 9, wherein each response further indicates a system age for the data storage node that sent the response.

11. A server according to claim 8, wherein each response further indicates a system load for the data storage node that sent the response.

12. A server according to claim 8, wherein the multicast storage query includes the data identifier for the data file.

13. A server according to claim 8, wherein at least three nodes are selected.

14. A server according to claim 8, wherein data storage nodes in a storage group are identified in a node list that is separate from the host list, the node list indicating Internet Protocol (IP) addresses of the data storage nodes in the node group.

15. A method for a device to update data in a data storage system, the device corresponding to a server accessing the data storage system or a data storage node in the data storage system, the method comprising:

the device sending a multicast storage query to a plurality of data storage nodes in the data storage system, wherein the multicast storage query includes a data identifier associated with a data file, and the multicast storage query indicates that the device is requesting a response for data storage nodes that stored a copy of the data file associated with the data identifier;

the device receiving responses to the multicast storage query from a subset of the plurality of data storage nodes, wherein each of the responses comprise a respective host list for the data file, the received host lists indicating data storage nodes in the data storage system that store copies of the data file;

the device sending a message comprising an updated version of the data file to the subset of data storage nodes that responded to the multicast storage query;

the device receiving acknowledgement information from one or more storage nodes in the subset, wherein the device determines from the acknowledgement information that at least a first data storage node in the subset successfully updated the data file and that at least a second data storage node in the subset did not successfully update the data file; and

the device sending an indication that a data replication procedure should be initiated based on determining that at least the second data storage node did not successfully update the data file, wherein the indication is sent to at least the first data storage node in order to trigger at least the first data storage node that did successfully update the file to perform the data replication procedure for the at least one file within the data storage system such that the data file is replicated to at least a third data storage node in the data storage system.

16. A method according to claim 15, wherein a data replication process is initiated based on the existence of multiple versions of the data file being stored in the data storage system.