Time-sensitive data storage operations in a dispersed storage network

• US 9,921,907 B2
• Filed: 02/09/2017
• Issued: 03/20/2018
• Est. Priority Date: 07/01/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Volvovski, Ilya, Khadiwala, Ravi, Dhuse, Greg, Resch, Jason K.
• Primary Examiner(s): Tran, Anh Tai
• Application Number:
  US15/428,390
• Publication Number:
  US 20170147428A1
• Time in Patent Office: 404 Days
• Field of Search: None
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a dispersed storage (DS) processing module transmitting a set of write commands for storing a set of encoded data slices in storage units of a dispersed storage network (DSN) and determining whether at least a first threshold number of write responses have been received within a response time period. When the at least the first threshold number of the write responses have been received within the response time period, the method continues with the DS processing module determining whether a total number of responses have been received within another response time period. When the total number of responses have not been received within the other response time period, the method continues with the DS processing module issuing a sub-set of write commit commands corresponding to a response number of encoded data slices for which a response was received.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the DSN including a plurality of storage units having memory for storing encoded data, the method comprises:

issuing a set of write commands for storing a set of encoded data slices in the plurality of storage units, wherein a data segment is dispersed storage error encoded into the set of encoded data slices, wherein the set of encoded data slices includes a total number of encoded data slices, wherein a decode threshold number of encoded data slices is needed to recover the data segment, and wherein the decode threshold number is less than the total number;

determining a timing profile, the timing profile including a plurality of incremental response time periods within a time frame following transmission of the set of write commands, wherein a respective threshold number of favorable write responses is associated with each of the incremental response time periods, wherein the respective threshold number associated with each incremental response time period is greater than the respective threshold number associated with a preceding incremental response time period, and wherein a favorable write response indicates that a corresponding encoded data slice of the set of encoded data slices has been stored, at least temporarily, in a storage unit of the plurality of storage units; and

when receiving at least the respective threshold number of favorable write responses associated with at least one of the incremental response time periods, issuing a write commit command for each of the encoded data slices associated with a favorable write response received during the at least one of the incremental response time periods, wherein the write commit command indicates that a corresponding encoded data slice is to be stored in the plurality of storage units on a non-temporary basis.

2. The method of claim 1, wherein the set of write commit commands includes at least a decode threshold number of write commit commands.

3. The method of claim 2, wherein the set of write commands further includes less than the total number of write commit commands.

4. The method of claim 1 further comprises:

when receiving less than a respective threshold number of favorable write responses associated with at least one of the incremental response time periods, issuing a write failure message to at least one of a requesting entity or a managing unit of the DSN.

5. The method of claim 1 further comprises:

when receiving less than the total number of write responses in a last response time period of the plurality of incremental response time periods, issuing a write failure message to at least one of a requesting entity or a managing unit of the DSN.

6. The method of claim 1, wherein a threshold number associated with a first response time period of the plurality of incremental response time periods is greater than the decode threshold number but less than the total number.

7. The method of claim 1, wherein a threshold number associated with a first response time period of the plurality of incremental response time periods is equal to the decode threshold number.

8. The method of claim 1 further comprises:

when receiving at least a decode number of write responses but less than a total number of write responses in a last response time period of the plurality of incremental response time periods;

determining identification information for one or more of the encoded data slices for which a write request was not received; and

storing the identification information in a rebuild queue.

9. The method of claim 1, wherein determining a timing profile further includes:

utilizing a pre-established response time period as a first response time period of the plurality of incremental response time periods; and

dynamically establishing a second response time period based, at least in part, on a number of favorable write responses received during the first response time period.

10. The method of claim 9 further comprises:

when receiving less than the total number of write responses during the first response time period, determining cost information associated with waiting for one or more additional write responses, wherein dynamically establishing the second response time period is further based on the cost information.

11. A computing device for use in a dispersed storage network (DSN), the DSN including a plurality of storage units having memory for storing encoded data, the computing device comprises:

a network interface;

a local memory; and

a processing module operably coupled to the network interface and the local memory, wherein the processing module is configured to;

issue, via the network interface, a set of write commands for storing a set of encoded data slices in the plurality of storage units, wherein a data segment is dispersed storage error encoded into the set of encoded data slices, wherein the set of encoded data slices includes a total number of encoded data slices, wherein a decode threshold number of encoded data slices is needed to recover the data segment, and wherein the decode threshold number is less than the total number;

determine a timing profile, the timing profile including a plurality of incremental response time periods within a time frame following transmission of the set of write commands, wherein a respective threshold number of favorable write responses is associated with each of the incremental response time periods, wherein the respective threshold number associated with each incremental response time period is greater than the respective threshold number associated with a preceding incremental response time period, and wherein a favorable write response indicates that a corresponding encoded data slice of the set of encoded data slices has been stored, at least temporarily, in a storage unit of the plurality of storage units; and

when receiving at least the respective threshold number of favorable write responses associated with at least one of the incremental response time periods, issue, via the network interface, a write commit command for each of the encoded data slices associated with a favorable write response received during the at least one of the incremental response time periods, wherein the write commit command indicates that a corresponding encoded data slice is to be stored in the plurality of storage units on a non-temporary basis.

12. The computing device of claim 11, wherein the set of write commit commands includes at least a decode threshold number of write commit commands.

13. The computing device of claim 12, wherein the set of write commands further includes less than the total number of write commit commands.

14. The computing device of claim 11, the processing module further configured to:

when receiving less than a respective threshold number of favorable write responses associated with at least one of the incremental response time periods, issue, via the network interface, a write failure message to at least one of a requesting entity or a managing unit of the DSN.

15. The computing device of claim 11, the processing module further configured to:

when receiving less than the total number of write responses in a last response time period of the plurality of incremental response time periods, issue, via the network interface, a write failure message to at least one of a requesting entity or a managing unit of the DSN.

16. The computing device of claim 11, wherein a threshold number associated with a first response time period of the plurality of incremental response time periods is greater than the decode threshold number but less than the total number.

17. The computing device of claim 11, wherein a threshold number associated with a first response time period of the plurality of incremental response time periods is equal to the decode threshold number.

18. The computing device of claim 11, the processing module further configured to:

when receiving at least a decode number of write responses but less than a total number of write responses in a last response time period of the plurality of incremental response time periods;

determine identification information for one or more of the encoded data slices for which a write request was not received; and

store the identification information in a rebuild queue.

19. The computing device of claim 11, the processing module further configured to:

utilize a pre-established response time period as a first response time period of the plurality of incremental response time periods; and

dynamically establish a second response time period based, at least in part, on a number of favorable write responses received during the first response time period.

20. The computing device of claim 19, the processing module further configured to:

when receiving less than the total number of write responses during the first response time period, determine cost information associated with waiting for one or more additional write responses, wherein dynamically establishing the second response time period is further based on the cost information.