Method for data retrieval from a distributed data storage system

• US 8,769,138 B2
• Filed: 09/02/2011
• Issued: 07/01/2014
• Est. Priority Date: 09/02/2011
• Status: Active Grant

• Current Assignee: Compuverde AB (Pure Storage, Inc.)
• Original Assignee: Compuverde AB (Pure Storage, Inc.)
• Inventors: Bernbo, Stefan, Melander, Christian, Persson, Roger, Petersson, Gustav
• Primary Examiner(s): Mejia, Anthony
• Application Number:
  US13/224,433
• Publication Number:
  US 20130060804A1
• Time in Patent Office: 1,033 Days
• Field of Search: 709/231, 709/200, 709/219, 709232-233, 711/171, 707/769
• US Class Current: 709/230
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

There is provided a method and server for retrieving data from a data storage system including a plurality of storage nodes. The method may include sending a multicast message to at least a subset of the storage nodes. The multicast message may include a request for the subset of storage nodes to send the data. The multicast message may further include a data identifier, indicating the data to be retrieved. Moreover, the method may include receiving data from a first storage node of the subset of storage nodes. The data received from the first storage node may correspond to the requested data. At least the act of sending a multicast message or the act of receiving data from the first storage node may be performed on a condition that an estimated size of the data is less than a predetermined value.

Claims

1. A device for retrieving data from a data storage system, the data storage system including a plurality of storage nodes, the device comprising:

a transmitter configured to send a multicast message requesting data to the plurality of storage nodes, wherein the multicast message includes a data identifier that identifies the requested data; and

a receiver configured to receive one of the requested data or a message including a reference to the requested data from a first storage node of said plurality of storage nodes after transmission of the multicast message, wherein;

the device is configured to receive the requested data in response to the multicast message rather than receiving the message including the reference to the requested data on condition that a size of the requested data is less than a predetermined value, and

the device is configured to receive the message including the reference to the requested data in response to the multicast message rather than directly receiving the requested data on condition that the size of the requested data exceeds the predetermined value, wherein the message including the reference to the requested data comprises an indication of one or more storage nodes that store the requested data; and

the transmitter is further configured to send a unicast communication requesting the requested data to at least one of the one or more storage nodes indicated in the message including the reference to the requested data on condition that the message including the reference to the requested data was received in response to the multicast message rather than the requested data being received in response to the multicast message.

2. The device according to claim 1, further comprising a processor configured to analyze the received data to determine if the data fulfills a set of criteria.

3. The device according to claim 2, wherein the processor is further configured to discard the received data based on the analysis.

4. The device according to claim 3, wherein the transmitter is configured to resend the multicast message to the plurality of storage nodes based on the analysis.

5. The device according to claim 1, wherein the receiver is further configured to receive the requested data from a second storage node of said plurality of storage nodes.

6. The device according to claim 5, wherein:

the receiver is further configured to receive the requested data from the first storage node; and

the device further comprises a processor configured to analyze the data received from the first storage node to determine if the data received from the first storage node fulfills a set of criteria, and analyze the data received from the second storage node to determine if the data received from the second storage node fulfills the set of criteria.

7. The device according to claim 6, wherein the processor is further configured to compare the data received from the first storage node to the data received from the second storage node using a checksum.

8. The device according to claim 7, wherein the processor is further configured to discard at least one of the data received from the first storage node or the data received from the second storage node based on the checksum.

9. The device according to claim 8, wherein the transmitter is configured to resend the multicast message based on the checksum.

10. The device according to claim 1, further comprising a processor configured to determine whether a capability of the first storage node fulfills a set of characteristics.

11. The device according to claim 1, wherein the data storage system is organized using a plurality of collection objects, each of the plurality of collection objects represent a directory or subdirectory within the data storage system, and a first collection object corresponding to a first directory comprises a reference to a second collection object corresponding to a subdirectory of the directory.

12. A method for sending data from a data storage system, the data storage system including a plurality of storage nodes, the method comprising:

receiving a multicast message from a server requesting data, wherein the multicast message includes a data identifier that identifies the requested data;

determining a size of the requested data; and

sending one of the requested data or a message including a reference to the requested data to the sever in response to receiving the multicast message, wherein the requested data is sent rather than sending the message including the reference to the requested data on condition that a size of the requested data is less than or equal to a predetermined value, the message including the reference to the requested data is sent rather than the requested data on condition that the size of the requested data exceeds the predetermined value, and the message including the reference to the requested data comprises an indication of one or more storage nodes that currently store the requested data.

13. The method according to claim 12, wherein the reference to the requested data comprises an indication of the size of the requested data, the data storage system is organized using a plurality of collection objects, each of the plurality of collection objects represent a directory or subdirectory within the data storage system, and a first collection object corresponding to a first directory comprises a reference to a second collection object corresponding to a subdirectory of the directory.

14. A server for retrieving data from a data storage system, the data storage system including a plurality of storage nodes, the server comprising:

a transmitter configured to send a multicast message that requests data to a plurality of storage nodes, wherein the server is configured to send the multicast message based on an estimated size of the data being less than a predetermined value, the multicast message includes a data identifier that identifies the requested data, the requested data comprises a collection object, the collection object represents a directory within the data storage system, and the collection object references one or more other collection objects that represent subdirectories within the data storage system; and

a receiver configured to receive the requested data from a first storage node of said plurality of storage nodes after transmission of the multicast message.

15. The server according to claim 14, further comprising a processor configured to analyze the data received from the first storage node to determine if the data fulfills a set of criteria.

16. The server according to claim 15, wherein the processor is further configured to discard the data received from the first storage node based on the analysis.

17. The server according to claim 16, wherein the transmitter is further configured to send the multicast message to the plurality of storage node again based on the analysis.

18. The server according to claim 14, wherein the receiver is further configured to receive the requested data from a second storage node of said plurality of storage nodes.

19. The server according to claim 18, further comprising a processor configured to analyze the data received from the first storage node to determine if the data received from the first storage node fulfills a set of criteria and analyze the data received from the second storage node to determine if the data received from the second storage node fulfills the set of criteria.

20. The server according to claim 19, wherein the processor is further configured to discard at least one of the data received from the first storage node or the data received from the second storage node based on the analysis.