Method for data maintenance

• US 10,430,443 B2
• Filed: 03/30/2018
• Issued: 10/01/2019
• Est. Priority Date: 09/02/2011
• Status: Active Grant

• Current Assignee: Compuverde AB (Pure Storage, Inc.)
• Original Assignee: Compuverde AB (Pure Storage, Inc.)
• Inventors: Bernbo, Stefan, Melander, Christian, Persson, Roger, Petersson, Gustav
• Primary Examiner(s): Truong, Lechi
• Application Number:
  US15/941,774
• Publication Number:
  US 20180225358A1
• Time in Patent Office: 550 Days
• Field of Search: 719310
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]

Abstract

A method for data storage implemented in a data storage system is disclosed. Data storage nodes may be interconnected by means of a communications network. The method may include sending a request for a first data item to a plurality of storage nodes. The first data item may include a reference to a second data item stored in the storage system. The method may include receiving the first data item from at least one storage node, and sending a request for the second data item to the plurality of storage nodes based on the reference included in the first data item.

Claims

1. A data storage node comprising a processor configured to:

identify a first data item in a data storage system, wherein the first data item comprises a first collection object corresponding to a directory of a distributed file system, the first collection object comprises one or more references to one or more other data items in the data storage system, and the one or more references comprise a reference to a second data item stored in the data storage system;

determine a multicast address used to request the second data item in a cluster of the data storage system based on the reference to the second data item included in the first data item;

send a multicast request for the second data item to the multicast address determined from the reference to the second data item included in the first data item;

receive one or more responses to the multicast request from one or more storage nodes;

send a unicast request for the second data item based on the one or more responses; and

receive the second data item from a storage node based on the unicast request.

2. The data storage node as in claim 1, wherein the second data item corresponds to a second collection object, and the second collection object corresponds to a first sub-directory of the directory.

3. The data storage node as in claim 2, wherein the processor is configured to add a new data file to the first sub-directory corresponding to the second collection object by adding a reference to the new data file to the second collection object.

4. The data storage node as in claim 2, wherein the processor is configured to add a second sub-directory to the directory corresponding to the first collection object by adding a reference to a third collection object to the first collection object, the third collection object corresponding to the second sub-directory.

5. The data storage node as in claim 2, wherein the processor is configured to delete the first sub-directory corresponding to the second collection object by removing the reference to the second collection object from the first collection object.

7. The server as in claim 1, wherein the second data item corresponds to a second collection object, and the second collection object corresponds to a first sub-directory of the directory.

8. The server as in claim 7, wherein the processor is configured to add a new data file to the first sub-directory corresponding to the second collection object by adding a reference to the new data file to the second collection object.

9. The server as in claim 7, wherein the processor is configured to add a second sub-directory to the directory corresponding to the first collection object by adding a reference to a third collection object to the first collection object, the third collection object corresponding to the second sub-directory.

10. The server as in claim 7, wherein the processor is configured to delete the first sub-directory corresponding to the second collection object by removing the reference to the second collection object from the first collection object.

6. A server comprising a processor configured to:

identify a first data item in a data storage system, wherein the first data item comprises a first collection object corresponding to a directory of a distributed file system, the first collection object comprises one or more references to one or more other data items in the data storage system, and the one or more references comprise a reference to a second data item stored in the data storage system;

determine a multicast address used to request the second data item in a cluster of the data storage system based on the reference to the second data item included in the first data item;

send a request for the second data item to the multicast address determined from the reference to the second data item included in the first data item; and

receive the second data item based on the request.

11. A method for implementing a distributed directory structure in a data storage system, the method comprising:

identifying a first data item in a data storage system, wherein the first data item comprises a first collection object corresponding to a directory of a distributed file system, the first collection object comprises one or more references to one or more other data items in the data storage system, and the one or more references comprise a reference to a second data item stored in the data storage system;

determining a multicast address used to request the second data item in a cluster of the data storage system based on the reference to the second data item included in the first data item;

sending a multicast request for the second data item to the multicast address determined from the reference to the second data item included in the first data item;

receiving one or more responses to the multicast request from one or more storage nodes;

sending a unicast request for the second data item based on the one or more responses; and

receiving the second data item from a storage node based on the unicast request.

12. The method as in claim 11, wherein multiple copies of the second data item are received based on the request, each of the multiple copies being received from a different data storage node of the cluster.

13. The method as in claim 11, wherein the first data item and the second data item are stored in different clusters of data storage nodes.

14. The method as in claim 11, wherein the second data item comprises a second collection object, the second collection object comprises at least one reference to at least one other data item in the data storage system, and the at least one reference comprises a reference to a third data item stored in the data storage system.

15. The method as in claim 14, wherein each of the first data item, the second data item, and the third data item are stored on different data storage nodes in the data storage system.

16. The method as in claim 11, wherein a collection object is a type of file utilized to implement the distributed directory structure.

17. The method as in claim 16, wherein collection objects in the data storage system each comprise at least one of a reference to another collection object or a reference to a data file.

18. The method as in claim 17, wherein the collection objects in the data storage system each represent a folder or subfolder of the distributed directory structure.

19. The method as in claim 11, wherein the first collection object also includes a reference to a data file, and the data file comprises payload data.

20. The method as in claim 11, wherein the first collection object is one of:

a binary file, a text file, or a data file.