Method for data maintenance

• US 8,645,978 B2
• Filed: 09/02/2011
• Issued: 02/04/2014
• Est. Priority Date: 09/02/2011
• Status: Expired due to Fees

• Current Assignee: Compuverde AB (Pure Storage, Inc.)
• Original Assignee: Compuverde AB (Pure Storage, Inc.)
• Inventors: Bernbo, Stefan, Melander, Christian, Persson, Roger, Petersson, Gustav
• Primary Examiner(s): Truong, Lechi
• Application Number:
  US13/224,415
• Publication Number:
  US 20130061252A1
• Time in Patent Office: 886 Days
• Field of Search: 719/328, 719/330
• US Class Current: 719/328
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]

Abstract

A method for data storage implemented in a data storage system is disclosed. Data storage nodes may be interconnected by means of a communications network. The method may include sending a request for a first data item to a plurality of storage nodes. The first data item may include a reference to a second data item stored in the storage system. The method may include receiving the first data item from at least one storage node, and sending a request for the second data item to the plurality of storage nodes based on the reference included in the first data item.

Claims

1. A method for maintaining data in a data storage system, the data storage system including data storage nodes interconnected by means of a communications network, the method comprising:

providing an application programming interface (API) to perform the following steps;

storing a root key;

identifying a first data item based on at least the root key;

sending a multicast request for a the first data item to a first of the plurality of storage nodes,

receiving the a first response from one or more storage nodes in the first of the plurality of storage nodes, wherein the first response comprises the first data item; and

determining a second data item based on a first reference in the first data item,

wherein the first reference in the first data item comprises an identifier for the second data item and a corresponding multicast address for locating the second data item;

sending a second request, wherein the second request requests the second data item, and the second request is sent using the corresponding multicast address;

receiving a second response, wherein the second response comprises the second data item;

determining a third data item based on a second reference in the second data item;

sending a third request for the third data item, wherein the third request is sent to an address identified in the second to reference; and

receiving the third data item, wherein the third data item comprises a data file.

2. The method according to claim 1, wherein the request for the first and second data entities are sent by multicast.

3. The method according to claim 1, wherein the first data item is associated with a first unique key and the second data item is associated with a second unique key.

4. The method according to claim 3, wherein each of the first unique key and second unique key comprises a cluster address and a data item identifier, wherein the cluster address identifies a subset of said storage nodes within the system and the data item identifier identifies a data item stored on the subset of storage nodes.

5. The method according to claim 4, further comprising sending the first and second request from an application programming interface (API).

6. The method according to claim 5, wherein the API is implemented on a server in communication with the storage system.

7. The method according to claim 5, wherein the API is implemented on a storage node within the storage system.

8. The method according to claim 5, wherein the first data item is received at the API, and the first unique key is identified in the API.

9. The method according to claim 1, wherein the data file comprises payload data.

10. The method according to claim 9, wherein the payload data is an image file.

11. The method according to claim 1, further comprising:

storing a fourth data item; and

sending an update for a fifth data item to another storage node, wherein the update for the fifth data item indicates that a reference to the fourth data item is to be included in the fifth data item.

12. The method according to claim 1, further comprising:

deleting a fourth data item; and

sending an update for a fifth data item to another storage node, wherein the update for the fifth data item indicates that a reference to the fourth data item included in the fifth data item is to be deleted.

13. The method according to claims 1, wherein the first data item is received by unicast message.

14. A data storage device in a data storage system that includes other data storage nodes interconnected via a communications network, the data storage device comprising:

a non-transitory storage medium configured to store a plurality of files;

a communication interface configured to communicate via the communications network;

and an application programming interface (API) configured to;

store one or more root keys in the storage medium,

determine a first data item based at least in part on at least one of the one or more root keys,

send a multicast request for the first data item to a first plurality of storage nodes via the communications interface,

receive a first response from one or more storage nodes in the first plurality of storage nodes via the communications interface, wherein the first response comprises the first data;

determine a second data item based on a first reference in the first data item,

wherein the first reference in the first data item comprises an identifier for the second data item and a corresponding multicast address for locating the second data item,

send a second request, wherein the second request requests the second data item, and the second request is sent using the corresponding multicast address,

receive a second response, wherein the second response comprises the second data item,

determine a third data item based on a second reference in the second data item;

sending a third request for the third data item, wherein the third request is sent to an address identified in the second reference; and

receive the third data item, wherein the third data item comprises a data file.

15. The data storage device according to claim 14, wherein the first data item is associated with a first unique key and the second data item is associated with a second unique key.

16. The data storage device according to claim 15, wherein the first unique key comprises a first cluster address and a first data item identifier, wherein the first cluster address identifies a first subset of said storage nodes within the system and the first data item identifier identifies a data item stored on the first subset of storage nodes.

17. A data storage node in a data storage system using a distributed directory system, the data storage node comprising:

a non-transitory storage medium configured to store a plurality of files;

a communication interface configured to communicate with other data storage nodes in the data storage system; and

an application program interface (API) configured to;

store a root key;

identify a first data item based on the root key;

send a multicast request for the first data item to a first plurality of storage nodes;

receive a first response from one or more storage nodes in the first plurality of storage nodes, wherein the first response comprises the first data item;

determine a second data item based on a first reference in the first data item, wherein the first reference in the first data item comprises an identifier for the second data item and a corresponding multicast address for locating the second data item; and

send a second request, wherein the second request requests the second data item, and the second request is sent using the corresponding multicast address,

receive a second response, wherein the second response comprises the second data item, determine a third data item based on a second reference in the second data item,

send a third request for the third data item, wherein the third request is sent to an address identified in the second reference, and

receive the third data item, wherein the third data item comprises a data file.

18. The data storage node according to claim 17, where the first request is sent to a multicast address identified by the root key.

19. The data storage node according to claim 17, wherein the API is further configured to determine that the second data item is a collection object based on a file structure of the second data item.