SCALING DISTRIBUTED QUEUES IN A DISTRIBUTED STORAGE NETWORK

• US 20190347005A1
• Filed: 05/11/2018
• Published: 11/14/2019
• Est. Priority Date: 05/11/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Lamba, Amit H., Scholl, Daniel J., Resch, Jason K., Ober, Brian F.
• Granted Patent: US 10,642,521 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method begins with a computing device of a distributed storage network (DSN) being designated to process a write request from a Decentralized, or Distributed, Agreement Protocol (DAP) and continues with the computing device retrieving a current policy for the storage units (SUs) associated with the write request, where the current policy specifies the current connectivity for the SUs. The method continues by using the current policy to configure a data structure to process the write request, where the data structure is based on scaling triggers and a load balancing scheme. The method continues with the data structure incorporating a primary queue and scaling the addition of secondary queues based on the scaling triggers and/or the load balancing scheme.

Claims

1. A method for execution by a computing device, the method comprises:

receiving, via a communication interface of the computing device, via a dispersed or distributed storage network (DSN), and from another computing device, a signal that specifies designation of the computing device to process a write request based on a system configuration of a Decentralized, or Distributed, Agreement Protocol (DAP);

receiving, via the communication interface, a request to process the write request based on the system configuration of the DAP;

retrieving, via the communication interface, a current policy of storage units (SUs) that are associated with the write request, wherein the current policy of the SUs specifies a current connectivity among the SUs within the DSN;

based on the current policy, configuring a data structure for processing the write request, wherein the configuring is based at least partially on one or more scaling triggers and further wherein the data structure includes at least a primary queue;

determining whether the primary queue compares favorably to the one or more scaling triggers;

in response to determining that the primary queue does not compare favorably to one or more scaling triggers, adding a first secondary queue to process the write request;

determining whether the primary queue and the first secondary queue together compare favorably to one or more scaling triggers; and

in response to determining that the primary queue and the first secondary queue do not compare favorably to one or more scaling triggers, adding a second secondary queue to process the write request.

2. The method of claim 1, wherein the one or more scaling triggers are selected from the group consisting of a system storage capacity, a delegate resource capacity, a number of items pressing in one or more queues, and a number of items pending to be entered into one or more queues.

3. The method of claim 2, wherein each of the one or more scaling triggers include an upper and a lower limit.

4. The method of claim 3, wherein the upper and lower limits are specified at a value chosen to prevent unfavorable addition and subtraction of one or more secondary queues.

5. The method of claim 1 further comprises:

after adding a second secondary queue determining, by the computing device, whether the primary queue and the first secondary queue compare favorably to one or more scaling triggers; and

in response to determining that the primary queue and the first secondary queue compare favorably to one or more scaling triggers, removing the second secondary from write request processing.

6. The method of claim 1, wherein the determining that the primary queue does not compare favorably to one or more scaling triggers is based on metrics associated with the primary queue.

7. The method of claim 6, wherein adding of the first secondary queue is further based on a comparison between the metrics associated with the primary queue and one or more of the one or more scaling triggers.

8. The method of claim 1, wherein the configuring the data structure for processing the write request is further based at least partially on a load balancing strategy.

9. The method of claim 8 further comprises:

receiving, via the communication interface of the computing device, metrics for use with the load balancing strategy, wherein the metrics are generated by a queue metrics engine (QME), wherein the QME tracks attributes of one or more queues, wherein the attributes are selected from a group consisting of queue tasks, queue performance, and queue reliability.

10. A computer readable memory comprises:

a first memory element that stores operational instructions, which, when executed by a first computing device of a dispersed storage network (DSN), causes the first computing device to;

receive, via a communication interface of the first computing device, via a dispersed or distributed storage network (DSN), and from another computing device, a signal that specifies designation of the computing device to process a write request based on a system configuration of a Decentralized, or Distributed, Agreement Protocol (DAP);

receive, via the communication interface, a request to process the write request based on the system configuration of the DAP;

retrieve, via the communication interface, a current policy of storage units (SUs) that are associated with the write request, wherein the current policy of the SUs specifies a current connectivity among the SUs within the DSN;

based on the current policy, configure a data structure for processing the write request, wherein the configuring is based at least partially on one or more scaling triggers and further wherein the data structure includes at least a primary queue;

determine whether the primary queue compares favorably to the one or more scaling triggers;

in response to determining that the primary queue does not compare favorably to one or more scaling triggers, add a first secondary queue to process the write request;

determine whether the primary queue and the first secondary queue together compare favorably to one or more scaling triggers; and

in response to determining that the primary queue and the first secondary queue do not compare favorably to one or more scaling triggers, add a second secondary queue to process the write request.

11. The computer readable memory of claim 10, wherein the one or more scaling triggers are selected from the group consisting of a system storage capacity, a delegate resource capacity, a number of items pressing in one or more queues, and a number of items pending to be entered into one or more queues.

12. The computer readable memory of claim 10, wherein each of the one or more scaling triggers include an upper and a lower limit.

13. The computer readable memory of claim 12, wherein the upper and lower limits are specified at a value chosen to prevent unfavorable addition and subtraction of one or more secondary queues.

14. The computer readable memory of claim 10, wherein the first memory element further stores operational instructions, which, when executed by the first computing device of a dispersed storage network (DSN), causes the first computing device to:

after adding a second secondary queue determine, by the computing device, whether the primary queue and the first secondary queue compare favorably to one or more scaling triggers; and

in response to determining that the primary queue and the first secondary queue compare favorably to one or more scaling triggers, remove the second secondary from write request processing.

15. The computer readable memory of claim 10, wherein the determining that the primary queue does not compare favorably to one or more scaling triggers is based on metrics associated with the primary queue.

16. The computer readable memory of claim 15, wherein adding of the first secondary queue is further based on a comparison between the metrics associated with the primary queue and one or more of the one or more scaling triggers.

17. The computer readable memory of claim 10, wherein the configuring the data structure for processing the write request is further based at least partially on a load balancing strategy.

18. The computer readable memory of claim 10, wherein the first memory element further stores operational instructions, which, when executed by the first computing device of a dispersed storage network (DSN), causes the first computing device to:

receive, via the communication interface of the computing device, metrics for use with a load balancing strategy, wherein the metrics are generated by a queue metrics engine (QME), wherein the QME tracks attributes of one or more queues, wherein the attributes are selected from a group consisting of queue tasks, queue performance, and queue reliability.

19. A computer program product for scaling queues in a distributed storage network (DSN), the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

receive, via a communication interface of the computing device, via the DSN, and from another computing device, a signal that specifies designation of the computing device to process a write request based on a system configuration of a Decentralized, or Distributed, Agreement Protocol (DAP);

receive, via the communication interface, a request to process the write request based on the system configuration of the DAP;

retrieve, via the communication interface, a current policy of storage units (SUs) that are associated with the write request, wherein the current policy of the SUs specifies a current connectivity among the SUs within the DSN;

based on the current policy, configure a data structure for processing the write request, wherein the configuring is based at least partially on one or more scaling triggers and further wherein the data structure includes at least a primary queue;

determine whether the primary queue compares favorably to the one or more scaling triggers;

in response to determining that the primary queue does not compare favorably to one or more scaling triggers, add a first secondary queue to process the write request;

determine whether the primary queue and the first secondary queue together compare favorably to one or more scaling triggers; and

in response to determining that the primary queue and the first secondary queue do not compare favorably to one or more scaling triggers, add a second secondary queue to process the write request.

20. The computer program product of claim 19, wherein the one or more scaling triggers are selected from the group consisting of a system storage capacity, a delegate resource capacity, a number of items pressing in one or more queues, and a number of items pending to be entered into one or more queues, and further wherein each of the one or more scaling triggers include an upper and a lower limit.