Storing an effective dynamic width of encoded data slices

• US 10,082,970 B2
• Filed: 02/20/2017
• Issued: 09/25/2018
• Est. Priority Date: 11/01/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew D., Dhuse, Greg R., Resch, Jason K., Volvovski, Ilya
• Primary Examiner(s): Shanmugasundaram, Kannan
• Application Number:
  US15/437,194
• Publication Number:
  US 20170160966A1
• Time in Patent Office: 582 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2212/7205 Cleaning, compaction, garbage collection, erase control
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/616 Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for storing a dynamic width of encoded data slices begins with determining dispersed storage error encoding parameters for encoding a data segment of data into a set of encoded data slices. The method continues by encoding the data segment into the set of encoded data slices and sending a set of write requests regarding the set of encoded data slices to a set of storage units. The method continues by receiving write responses from at least some storage units and when at least a write threshold number of favorable write responses have been received within a first time period, by sending a set of write commit requests to the at least some storage units. When less than an initial dynamic width number of favorable write responses have been received, the method continues by flagging a rebuild number of encoded data slices.

Claims

1. A method for execution by a computing device of a dispersed storage network (DSN), the method comprises:

determining dispersed storage error encoding parameters for encoding a data segment of data into a set of encoded data slices, wherein the dispersed storage error encoding parameters include a decode threshold number, a write threshold number, a pillar width number, and an initial dynamic width number;

encoding the data segment into the set of encoded data slices, wherein the set of encoded data slices includes the pillar width number of encoded data slices;

sending a set of write requests regarding the set of encoded data slices to a set of storage units of the DSN;

receiving write responses from at least some storage units of the set of storage units; and

when at least the write threshold number of favorable write responses have been received within a first time period;

sending a set of write commit requests to the at least some storage units; and

when less than the initial dynamic width number of favorable write responses have been received, flagging a rebuild number of encoded data slices of the set of encoded data slices, wherein the rebuild number is substantially equal to a difference between the initial dynamic width number and an actual number of favorable write responses received.

2. The method of claim 1 further comprises:

determining the initial dynamic width by one or more of;

storage requirements;

desired reliability;

a lookup;

a query;

historical performance of the set of storage units;

data storage request message content; and

a message.

3. The method of claim 1, wherein the sending the set of write requests further comprises:

sending the pillar width number of write requests to the pillar width number of storage units of the set of storage units.

4. The method of claim 1, wherein the sending the set of write requests further comprises:

sending the initial dynamic width number of write requests to the initial dynamic width number of storage units of the set of storage units.

5. The method of claim 4 further comprises:

when the at least the write threshold number of favorable write responses have not been received within the first time period;

increasing the initial dynamic width number; and

sending another set of write requests to the increased dynamic width number of storage units of the set of storage units.

6. The method of claim 1 further comprises:

sending a set of read requests for the set of encoded data slices to a current dynamic width number of storage units of the set of storage units.

7. The method of claim 6 further comprises:

selecting an optimal number of storage units of the current dynamic width number of storage units for the sending the set of read requests, wherein the optimal number is based on one or more of;

latency;

bandwidth;

reliability;

availability;

security; and

cost.

8. The method of claim 1 further comprises:

when more than the initial dynamic width number and less than the pillar width number of favorable write responses have been received, determining whether to rebuild at least one of a missing number of encoded data slices, wherein the missing number is substantially equal to a difference between the pillar width number and an actual number of favorable write responses received; and

when determining to rebuild the at least one of the missing number of encoded data slices;

rebuilding the at least one of the missing number of encoded data slices; and

sending write requests regarding the at least one of the missing number of encoded data slices to other storage units of the set of storage units; and

when determining not to rebuild the at least one of the missing number of encoded data slices;

determining whether to delete at least one of an excess number of encoded data slices, wherein the excess number is substantially equal to a difference between the initial dynamic width number and the actual number of favorable write responses received; and

when determining to delete the at least one of the excess number of encoded data slices;

sending delete requests regarding the at least one of the excess number of encoded data slices to the at least some storage units of the set of storage units.

9. A computing device of a dispersed storage network (DSN), wherein the computing device comprises:

memory;

an interface; and

a processing module operably coupled to the memory and the interface, wherein the processing module is operable to;

determine dispersed storage error encoding parameters for encoding a data segment of data into a set of encoded data slices, wherein the dispersed storage error encoding parameters include a decode threshold number, a write threshold number, a pillar width number, and an initial dynamic width number;

encode the data segment into the set of encoded data slices, wherein the set of encoded data slices includes the pillar width number of encoded data slices;

send a set of write requests regarding the set of encoded data slices to a set of storage units of the DSN;

receive write responses from at least some storage units of the set of storage units; and

when at least the write threshold number of favorable write responses have been received within a first time period;

send a set of write commit requests to the at least some storage units; and

when less than the initial dynamic width number of favorable write responses have been received, flag a rebuild number of encoded data slices of the set of encoded data slices, wherein the rebuild number is substantially equal to a difference between the initial dynamic width number and an actual number of favorable write responses received.

10. The computing device of claim 9, wherein the processing module is further operable to:

determine the initial dynamic width by one or more of;

storage requirements;

desired reliability;

a lookup;

a query;

historical performance of the set of storage units;

data storage request message content; and

a message.

11. The computing device of claim 9, wherein the processing module is further operable to send the set of write requests by:

sending the set of write requests to a number of storage units of the set of storage units, wherein the number is the pillar width number.

12. The computing device of claim 9, wherein the processing module is further operable to send the set of write requests by:

sending the set of write requests to a number of storage units of the set of storage units, wherein the number is the initial dynamic width number.

13. The computing device of claim 9, wherein the processing module is further operable to:

when the at least the write threshold number of favorable write responses have not been received within the first time period;

increase the initial dynamic width number; and

send another set of write requests to the increased dynamic width number of storage units of the set of storage units.

14. The computing device of claim 9, wherein the processing module is further operable to:

send a set of read requests for the set of encoded data slices to a current dynamic width number of storage units of the set of storage units.

15. The computing device of claim 14, wherein the processing module is further operable to:

select an optimal number of storage units of the current dynamic width number of storage units for sending the set of read requests, wherein the optimal number is based on one or more of;

latency;

bandwidth;

reliability;

availability;

security; and

cost.

16. The computing device of claim 9, wherein the processing module is further operable to:

when more than the initial dynamic width number and less than the pillar width number of favorable write responses have been received;

determine whether to rebuild at least one missing number of encoded data slices, wherein the missing number is substantially equal to a difference between the pillar width number and an actual number of favorable write responses received; and

when determining to rebuild the at least one missing number of encoded data slices;

rebuild the at least one missing number of encoded data slices; and

sending write requests regarding the at least one missing number of encoded data slices to other storage units of the set of storage units; and

when determining not to rebuild the at least one of the missing number of encoded data slices;

determining whether to delete at least one of an excess number of encoded data slices, wherein the excess number is substantially equal to a difference between the initial dynamic width number and the actual number of favorable write responses received; and

when determining to delete the at least one of the excess number of encoded data slices;

sending delete requests regarding the at least one of the excess number of encoded data slices to the at least some storage units of the set of storage units.