Reprioritizing pending dispersed storage network requests

• US 9,632,872 B2
• Filed: 04/19/2013
• Issued: 04/25/2017
• Est. Priority Date: 06/05/2012
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Dhuse, Greg, Volvovski, Ilya, Baptist, Andrew
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): RUSSELL, ANDREW D
• Application Number:
  US13/866,457
• Publication Number:
  US 20130326167A1
• Time in Patent Office: 1,467 Days
• Field of Search: 711158
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16

Abstract

A method begins by a dispersed storage (DS) processing module monitoring processing status of a plurality of pending dispersed storage network (DSN) access requests, where less than a desired number of DS units have favorably responded to a set of access requests. The method continues with the DS processing module interpreting the processing status of the plurality of pending DSN access requests to detect a processing anomaly. The method continues with the DS processing module reprioritizing further processing of at least one of the plurality of pending DSN access requests having the processing anomaly and another one or more of the plurality of pending DSN access requests. The method continues with the DS processing module sending notice of the reprioritized further processing to one or more DS units.

Claims

1. A method for execution by computing device within a dispersed storage network (DSN), the method comprising:

monitoring processing status of a plurality of pending DSN access requests, wherein a pending DSN access request of the plurality of pending DSN access requests includes a set of access requests regarding a set of encoded data slices, wherein the set of access requests is sent to dispersed storage (DS) units of the DSN, wherein less than a desired number of the DS units have favorably responded to the set of access requests, wherein a data segment of data is dispersed storage error encoded to produce the set of encoded data slices, wherein the desired number of DS units corresponds to a decode threshold number of encoded data slices of the set of encoded data slices, and wherein the decode threshold number of encoded data slices are required to recover a data segment;

interpreting the processing status of the plurality of pending DSN access requests to detect a processing anomaly of one of the plurality of pending DSN access requests;

reprioritizing further processing of at least one of;

the one of the plurality of pending DSN access requests having the processing anomaly;

or

another one or more of the plurality of pending DSN access requests; and

sending notice of the reprioritized further processing to one or more of the DS units.

2. The method of claim 1, wherein the monitoring the processing status comprises one or more of:

determining a number of favorable replies received from the DS units regarding the pending DSN access request;

determining a rate of receiving favorable replies from the DS units regarding the pending DSN access request;

determining a priority of the pending DSN access request;

determining a request type of the pending DSN access request;

determining that at least a threshold number of favorable replies from the DS units regarding the pending DSN access request have been received;

or

determining a duration of the pending DSN access request.

3. The method of claim 1, wherein the interpreting the processing status comprises at least one of:

indicating the processing anomaly when a desired number of favorable replies from the DS units regarding the one of the plurality of pending DSN access requests has not been received;

indicating the processing anomaly when a desired rate of receiving favorable replies from the DS units regarding the one of the plurality of pending DSN access requests is not achieved;

indicating the processing anomaly when the one of the plurality of pending DSN access requests has been pending for longer than a desired duration;

or

indicating the processing anomaly when a threshold number of favorable replies from the DS units regarding the one of the plurality of pending DSN access requests have been received.

4. The method of claim 1, wherein the processing anomaly comprises one of:

processing of the one of the plurality of pending DSN access requests is exceeding a first desired processing level; and

the processing of the one of the plurality of pending DSN access requests is below a second desired processing level.

5. The method of claim 1, wherein the reprioritizing the further processing comprises:

decreasing priority of the other one or more of the plurality of pending DSN access requests when the processing of the other one or more of the plurality of pending DSN access requests is above a first desired processing level; and

increasing priority of the one of the plurality of pending DSN access requests having the processing anomaly, when the processing anomaly is indicative of processing of the one of the plurality of pending DSN access requests being below a second desired processing level.

6. The method of claim 1, wherein the reprioritizing the further processing comprises:

increasing priority of the one of the plurality of pending DSN access requests having the processing anomaly.

7. The method of claim 1, wherein the reprioritizing the further processing comprises:

decreasing priority of the other one or more of the plurality of pending DSN access requests.

8. The method of claim 1, wherein the sending the notice of the reprioritized further processing comprises:

identifying the one or more of the DS units as units of the DS units that have not provided a reply to the one of the plurality of pending DSN access requests or to the other one or more of the plurality of pending DSN access requests.

9. The method of claim 1, wherein the pending DSN access request comprises one or more of:

a read request;

a request of a three-phase write commit process;

a status request;

a delete request;

or

an update request.

10. A dispersed storage (DS) module comprising:

a first processing module, when operable within a computing device, causes the computing device to;

monitor processing status of a plurality of pending dispersed storage network (DSN) access requests, wherein a pending DSN access request of the plurality of pending DSN access requests includes a set of access requests regarding a set of encoded data slices, wherein the set of access requests is sent to dispersed storage (DS) units of the DSN, wherein less than a desired number of the DS units have favorably responded to the set of access requests, wherein a data segment of data is dispersed storage error encoded to produce the set of encoded data slices, wherein the desired number of DS units corresponds to a decode threshold number of encoded data slices of the set of encoded data slices, and wherein the decode threshold number of encoded data slices are required to recover a data segment;

a second processing module, when operable within the computing device, causes the computing device to;

interpret the processing status of the plurality of pending DSN access requests to detect a processing anomaly of one of the plurality of pending DSN access requests;

a third processing module, when operable within the computing device, causes the computing device to;

reprioritize further processing of at least one of;

the one of the plurality of pending DSN access requests having the processing anomaly;

or

another one or more of the plurality of pending DSN access requests; and

a fourth processing module, when operable within the computing device, causes the computing device to;

send notice of the reprioritized further processing to one or more of the DS units.

11. The DS module of claim 10, wherein the first processing module functions to monitor the processing status by one or more of:

determining a number of favorable replies received from the DS units regarding the pending DSN access request;

determining a rate of receiving favorable replies from the DS units regarding the pending DSN access request;

determining a priority of the pending DSN access request;

determining a request type of the pending DSN access request;

determining that at least a threshold number of favorable replies from the DS units regarding the pending DSN access request have been received;

or

determining a duration of the pending DSN access request.

12. The DS module of claim 10, wherein the second processing module functions to interpret the processing status by at least one of:

indicating the processing anomaly when a desired number of favorable replies from the DS units regarding the one of the plurality of pending DSN access requests has not been received;

indicating the processing anomaly when a desired rate of receiving favorable replies from the DS units regarding the one of the plurality of pending DSN access requests is not achieved;

indicating the processing anomaly when the one of the plurality of pending DSN access requests has been pending for longer than a desired duration;

or

indicating the processing anomaly when a threshold number of favorable replies from the DS units regarding the one of the plurality of pending DSN access requests have been received.

13. The DS module of claim 10, wherein the processing anomaly comprises one of:

processing of the one of the plurality of pending DSN access requests is exceeding a first desired processing level; and

the processing of the one of the plurality of pending DSN access requests is below a second desired processing level.

14. The DS module of claim 10, wherein the third processing module functions to reprioritize the further processing by:

decreasing priority of the other one or more of the plurality of pending DSN access requests when the processing of the other one or more of the plurality of pending DSN access requests is above a first desired processing level; and

increasing priority of the one of the plurality of pending DSN access requests having the processing anomaly, when the processing anomaly is indicative of processing of the one of the plurality of pending DSN access requests being below a second desired processing level.

15. The DS module of claim 10, wherein the third processing module functions to reprioritize the further processing by:

increasing priority of the one of the plurality of pending DSN access requests having the processing anomaly.

16. The DS module of claim 10, wherein the third processing module functions to reprioritize the further processing by:

decreasing priority of the other one or more of the plurality of pending DSN access requests.

17. The DS module of claim 10, wherein the fourth processing module functions to send the notice of the reprioritized further processing by:

identifying the one or more of the DS units as units of the DS units that have not provided a reply to the one of the plurality of pending DSN access requests or to the other one or more of the plurality of pending DSN access requests.

18. The DS module of claim 10, wherein the pending DSN access request comprises one or more of:

a read request;

a request of a three-phase write commit process;

a status request;

a delete request;

or

an update request.