Method and device for maintaining data in a data storage system comprising a plurality of data storage nodes

• US 8,843,710 B2
• Filed: 12/18/2013
• Issued: 09/23/2014
• Est. Priority Date: 09/02/2011
• Status: Active Grant

• Current Assignee: Compuverde AB (Pure Storage, Inc.)
• Original Assignee: Compuverde AB (Pure Storage, Inc.)
• Inventors: Bernbo, Stefan, Melander, Christian, Persson, Roger, Petersson, Gustav
• Primary Examiner(s): Chery, Mardochee
• Application Number:
  US14/133,540
• Publication Number:
  US 20140108745A1
• Time in Patent Office: 279 Days
• Field of Search: 711/151
• US Class Current: 711/151
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method and device for maintaining data in a data storage system, comprising a plurality of data storage nodes, the method being employed in a storage node in the data storage system and comprising: monitoring and detecting, conditions in the data storage system that imply the need for replication of data between the nodes in the data storage system; initiating replication processes in case such a condition is detected, wherein the replication processes include sending multicast and unicast requests to other storage nodes, said requests including priority flags, receiving multicast and unicast requests from other storage nodes, wherein the received requests include priority flags, ordering the received requests in different queues depending on their priority flags, and dealing with requests in higher priority queues with higher frequency than requests in lower priority queues.

Claims

1. A data storage system comprising:

a plurality of data storage nodes, wherein each data storage node of the plurality of data storage nodes is configured to;

send requests to other data storage nodes in the data storage system;

receive requests from at least one client device external to the data storage system and from the other data storage nodes in the data storage system, wherein at least three received requests comprise a first request, a second request, and a third request, the first request indicates a first priority flag, the second request indicates a second priority flag, and the third request indicates a third priority flag;

order the received requests in a plurality of priority queues based on the first, second, and third priority flags, wherein requests that include the first priority flag correspond to external client requests from the at least one client device and are included in a highest priority queue, requests that include the second or third priority flags correspond to requests from one or more of the other data storage nodes in the data storage system, requests that include the second priority flag are included in a higher priority queue, and requests that include the third priority flag are included in a lower priority queue; and

concurrently process the plurality of priority queues according to defined frequencies, wherein requests in the highest priority queue are processed with a first frequency, requests in the higher priority queue are processed with a second frequency, requests in the lower priority queue are processed with a third frequency, the first frequency is greater than the second frequency, and the second frequency is greater than the third frequency.

2. The data storage system as in claim 1, wherein each of the plurality of data storage nodes maintain one or more host lists that each indicate one or more other data storage nodes that store a specific data entity.

3. The data storage system as in claim 1, wherein each of the plurality of data storage nodes in the data storage system is configured to initiate replication of data within the data storage system.

4. The data storage system as in claim 1, wherein the data storage system does not include a centralized control node for controlling data replication among the plurality of data storage nodes in the data storage system.

5. A data storage node of a data storage system, the data storage node comprising:

a receiver configured to receive a first request from a client device external to the data storage system, wherein the first request is at least one of a multicast or unicast request and includes a first priority flag;

the receiver further configured to receive a second request from a second data storage node of the data storage system, wherein the second request is at least one of a multicast or unicast request and includes a second priority flag;

the receiver further configured to receive a third request from a third data storage node of the data storage system, wherein the third request is at least one of a multicast or unicast request and includes a third priority flag;

a queue unit configured to order the requests in a plurality of priority queues based on the first, second, and third priority flags, wherein requests that include the first priority flag are included in a highest priority queue reserved for external client requests, requests that include the second priority flag are included in a higher priority queue for data storage system internal requests, and request that include the third priority flag are included in a lower priority queue for data storage system internal requests; and

a processor configured to concurrently process the plurality of priority queues according to defined frequencies, wherein requests in the highest priority queue are processed with a first frequency, requests in the higher priority queue are processed with a second frequency, requests in the lower priority queue are processed with a third frequency, the first frequency is greater than the second frequency, and the second frequency is greater than the third frequency.

6. The data storage node as in claim 5, further comprising

a monitoring unit configured to monitor the plurality of data storage nodes;

a detecting unit configured to detect a condition in the data storage system, the condition indicating that data stored on at least one of the plurality of data storage nodes should be replicated within the data storage system; and

a replication unit configured to initiate a replication process based on detecting the condition, wherein the replication process comprises sending at least one of a multicast or unicast request to at least one data storage node of the plurality of data storage nodes, said request including a respective priority flag indicative of a relative priority for the replication of the data.

7. The data storage node as in claim 5, wherein the first frequency is a multiple of the second frequency.

8. The data storage node as in claim 5, wherein the higher priority queue corresponds to requests to replicate a data file for which there is a single copy currently stored in the data storage system.

9. The data storage node as in claim 5, wherein the higher priority queue is associated with a maximum queue size and requests including the second priority flag are rejected upon reaching the maximum queue size.

10. The data storage node as in claim 9, further comprising a transmitter configured to transmit a busy message to at least one storage node that sent a request corresponding to a queue that has reached its maximum queue size.

11. The data storage node as in claim 5, wherein the first request comprises a request by the client device external to the data storage system to write data in the data storage system.

12. The data storage node as in claim 5, wherein each data storage node in the data storage node of the plurality of data storages is configured to transmit requests to and receive requests from other data storage nodes in the data storage system.

13. A method for maintaining data in a data storage system comprising a plurality of data storage nodes, the method comprising:

receiving a first request from a client device external to the data storage system, wherein the first request is at least one of a multicast or unicast request and includes a first priority flag;

receiving a second request from a second request from a second data storage node of the data storage system, wherein the second request is at least one of a multicast or unicast request and includes a second priority flag;

receiving a third request from a third data storage node of the data storage system, wherein the third request is at least one of a multicast or unicast request and includes a third priority flag;

ordering the requests in a plurality of priority queues based on the first, second, and third priority flags, wherein requests that include the first priority flag are included in a highest priority queue reserved for external client requests, requests that include the second priority flag are included in a higher priority queue for data storage system internal requests, and request that include the third priority flag are included in a lower priority queue for data storage system internal requests; and

concurrently processing the plurality of priority queues at relative frequencies, wherein requests in the highest priority queue are processed with higher frequency than requests in the higher priority queue, and requests in the higher priority queue are processed with higher frequency than requests in the lower priority queue.

14. The method as in claim 13, wherein the lower priority queue is associated with a maximum queue size and requests including the third priority flag are rejected upon reaching the maximum queue size.

15. The method as in claim 14, further comprising transmitting a busy message to at least one storage node that sent a request corresponding to a queue that has reached its maximum queue size.

16. The method as in claim 13, wherein a length of the higher priority queue is longer than a length of the lower priority queue.

17. The method as in claim 13, wherein the first request corresponds to a request to update a data file stored in the data storage system.

18. The method as in claim 17, wherein the first request from the client device external to the data storage system is received from a server accessing the data storage system.

19. The method as in claim 13, wherein each of the first and second requests comprises at least one of a request to read data in the data storage systems or a request to write data in the data storage system.

20. The method as in claim 13, wherein a first processing frequency is used for the higher priority queue, a second processing frequency is used for the lower priority queue, and the first processing frequency is a multiple of the second processing frequency.