Prioritization of messages of a dispersed storage network

• US 10,469,578 B2
• Filed: 11/21/2012
• Issued: 11/05/2019
• Est. Priority Date: 11/28/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Dhuse, Greg
• Primary Examiner(s): Ndiaye, Cheikh T
• Assistant Examiner(s): Habtemariam, Melaku Y
• Application Number:
  US13/683,951
• Publication Number:
  US 20130138756A1
• Time in Patent Office: 2,540 Days
• Field of Search: 709201-203, 709207, 709217-219, 709232, 709240
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 12/0607 Interleaved addressing
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 65/10 Architectures or entities
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 9/0825 using asymmetric-key encryption or public key infrastructure [PKI], e.g. key signature or public key certificates
• CPC Class Codes: H04L 9/3236 using cryptographic hash functions

Abstract

A method begins by a dispersed storage (DS) processing module generating a set of messages regarding a set of encoded data slices. For a first message of the set of messages, the method continues with the DS processing module determining a first message priority based on system-level message processing information and message processing status of a first storage node. For a second message of the set of messages, the method continues with the DS processing module determining a second message priority based on the system-level message processing information and message processing status of a second storage node. The method continues with the DS processing module sending the first message of the set of messages to the first storage node in accordance with the first message priority and sending the second message of the set of messages to the second storage node in accordance with the second message priority.

Claims

1. A method for execution by a computing device, the method comprises:

storing a plurality of sets of messages in a set of queues corresponding to a set of storage nodes, wherein a set of messages of the plurality of sets of messages is associated with a set of encoded data slices, wherein a data segment of data is encoded using a dispersed storage error coding function to produce the set of encoded data slices and wherein a queue of the set of queues stores a group of messages that includes a message from each of at least some of the plurality of sets of messages for a particular storage unit of the set of storage nodes;

on a queue by queue basis, prioritizing the group of messages based on a queue prioritization scheme;

as the plurality of sets of messages are processed by the set of storage nodes, updating, on the queue by queue basis, the prioritizing of the group of messages based on the processing of the plurality of sets of messages by the set of storage nodes;

for a message of a first group of messages, sending the message of the first group of messages to a first storage node of the set of storage nodes in accordance with the prioritizing of the group of messages for the first storage node; and

for a message of a second group of messages, sending the message of the second group of messages to a second storage node of the set of storage nodes in accordance with the prioritizing of the group of messages for the second storage node.

2. The method of claim 1 further comprises:

the set of messages including a set of write messages to write the set of encoded data slices to a dispersed storage network;

determining when a write threshold number of the set of write messages have been sent; and

when the write threshold number of the set of write messages have been sent, for write messages of the set of write messages that have not yet been processed, adjusting the prioritizing.

3. The method of claim 2, wherein the adjusting the prioritizing comprises one of:

changing one of the write messages of the set of write messages that have not yet been processed;

deleting the one of the write messages of the set of write messages that have not yet been processed; and

reducing priority of the one of the write messages of the set of write messages that have not yet been processed.

4. The method of claim 1 further comprises:

the set of messages including a set of read messages to read the set of encoded data slices;

determining when a decode threshold number of the set of read messages have been received; and

when the decode threshold number of the set of read messages have been received, for read messages of the set of read messages that have not yet been processed, adjusting the prioritizing.

5. The method of claim 4, wherein the adjusting the prioritizing comprises one of:

changing one of the read messages of the set of read messages that have not yet been processed;

deleting the one of the read messages of the set of read messages that have not yet been processed; and

reducing priority of the one of the read messages of the set of read messages that have not yet been processed.

6. The method of claim 1, wherein the prioritizing comprises:

for a first set of messages of the plurality of sets of messages, determining at least one of;

a current status of sending the first set of messages to the set of storage nodes, and a current status of successfully processing the first set of messages by the set of storage nodes; and

for a second set of messages of the plurality of sets of messages, determining at least one of;

a current status of sending the second set of messages to the set of storage nodes, and a current status of successfully processing the second set of messages by the set of storage nodes.

7. The method of claim 1, wherein updating the prioritizing of a first queue of the set of queues comprises:

determining a number of sets of the plurality of sets of messages that involves the first storage node;

determining status of sending messages of the number of sets of the plurality of sets of messages to the first storage node;

determining status of successfully processed messages of the number of sets of the plurality of sets of messages by the first storage node; and

determining the message processing status of the first storage node based on the status of sending messages and the status of successfully processed messages.

8. The method of claim 1 further comprises:

the set of messages including a set of write messages to write the set of encoded data slices;

determining that a write threshold number of the set of write messages have not been sent to the set of storage nodes within a given time frame; and

when the write threshold number of the set of write messages have not been sent in the given time frame, increasing the prioritizing.

9. The method of claim 1 further comprises:

the set of messages including a set of read messages to read the set of encoded data slices;

determining when a decode threshold number of the set of read messages have not been sent within a given time period; and

when the decode threshold number of the set of read messages have not been sent within the given time period, increasing the prioritizing.

10. A dispersed storage (DS) hardware module comprises:

a first hardware module, when operable within a computing device, causes the computing device to;

store a plurality of sets of messages in a set of queues corresponding to a set of storage nodes, wherein a set of messages of the plurality of sets of messages is associated with a set of encoded data slices, wherein a data segment of data is encoded using a dispersed storage error coding function to produce the set of encoded data slices and wherein a queue of the set of queues stores a group of messages that includes a message from each of at least some of the plurality of sets of messages for a particular storage unit of the set of storage nodes;

a second module, when operable within the computing device, causes the computing device to;

on a queue by queue basis, prioritizing the group of messages based on a queue prioritization scheme; and

as the plurality of sets of messages are processed by the set of storage nodes, updating, on the queue by queue basis, the prioritizing of the group of messages based on the processing of the plurality of sets of messages by the set of storage nodes;

a fourth module, when operable within the computing device, causes the computing device to;

for a message of a first group of messages, send the message of the first group of messages to a first storage node of the set of storage nodes in accordance with the prioritizing of the group of messages for the first storage node; and

for a message of a second group of messages, send the message of the second group of messages to a second storage node of the set of storage nodes in accordance with the prioritizing of the group of messages for the second storage node.

11. The DS module of claim 10 further comprises:

the set of messages including a set of write messages to write the set of encoded data slices to a dispersed storage network; and

the fourth module further functions to determine when a write threshold number of the set of write messages have been sent; and

when the write threshold number of the set of write messages have been sent, for write messages of the set of write messages that have not yet been processed, the second module further functions to adjust the prioritizing.

12. The DS module of claim 11, wherein the adjusting the prioritizing comprises one of:

changing one of the write messages of the set of write messages that have not yet been processed;

deleting the one of the write messages of the set of write messages that have not yet been processed; and

reducing priority of the one of the write messages of the set of write messages that have not yet been processed.

13. The DS module of claim 10 further comprises:

the set of messages including a set of read messages to read the set of encoded data slices;

the fourth module further functions to determine when a decode threshold number of the set of read messages have been sent; and

when the decode threshold number of the set of read messages have been sent, for read messages of the set of read messages that have not yet been sent, the second module further functions to adjust the prioritizing.

14. The DS module of claim 13, wherein the adjusting the prioritizing comprises one of:

changing one of the read messages of the set of read messages that have not yet been processed;

deleting the one of the read messages of the set of read messages that have not yet been processed; and

reducing priority of the one of the read messages of the set of read messages that have not yet been processed.

15. The DS module of claim 10, wherein the second module functions to prioritize by:

for a first set of messages of the plurality of sets of messages, determining at least one of;

a current status of sending the first set of messages to the set of storage nodes, and a current status of successfully processing the first set of messages by the set of storage nodes; and

for a second set of messages of the plurality of sets of messages, determining at least one of;

a current status of sending the second set of messages to the set of storage nodes, and a current status of successfully processing the second set of messages by the set of storage nodes.

16. The DS module of claim 10 further comprises:

the second module further functions to update the prioritizing of a first queue of the set of queues by;

determining a number of sets of the plurality of sets of messages that involves the first storage node;

determining status of sending messages of the number of sets of the plurality of sets of messages to the first storage node;

determining status of successfully processed messages of the number of sets of the plurality of sets of messages by the first storage node; and

determining the message processing status of the first storage node based on the status of sending messages and the status of successfully processed messages.

17. The DS module of claim 10 further comprises:

the set of messages including a set of write messages to write the set of encoded data slices;

the fourth module further functions to determine that a write threshold number of the set of write messages have not been sent within a given time frame; and

when the write threshold number of the set of write messages have not been sent in the given time frame, the second module further functions to increase the prioritizing.

18. The DS module of claim 10 further comprises:

the set of messages including a set of read messages to read the set of encoded data slices;

the fourth module further functions to determine when a decode threshold number of the set of read messages have not been sent within a given time period; and

when the decode threshold number of the set of read messages have not been sent within the given time period, the second module further functions to increase the prioritizing.