Accessing common data in a dispersed storage network

• US 10,037,171 B2
• Filed: 03/02/2016
• Issued: 07/31/2018
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew, Abhijeet, Kumar, Resch, Jason K.
• Primary Examiner(s): Bryan, Jason B
• Application Number:
  US15/058,408
• Publication Number:
  US 20160321136A1
• Time in Patent Office: 881 Days
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method begins by a computing device of a dispersed storage network (DSN) encoding digital content into a plurality of sets of encoded data slices and generating at least one set of write commands, where a write command includes an indication of a number of copies of an encoded data slice to be stored. The method continues by a set of storage units of the DSN storing the plurality of sets of encoded data slices, where a storage unit stores the number of copies of the encoded data slice in accordance with the write command. The method continues by the set of storage units receiving a plurality of retrieval requests from a plurality of requesting computing devices regarding the digital content and fulfilling the plurality of retrieval requests utilizing various combinations of copies of encoded data slices of the plurality of sets of encoded data slices.

Claims

1. A method comprises:

encoding, by a first computing device of a dispersed storage network (DSN), digital content into a plurality of sets of encoded data slices;

generating, by the first computing device, at least one set of write commands regarding the plurality of sets of encoded data slices, wherein a write command of the at least one set of write commands is regarding an encoded data slice of a set of the plurality of sets of encoded data slices and includes an indication of a number of copies of the encoded data slice to be stored, wherein the write command is generated to include a set of slice names for the copies of the encoded data slices, and wherein each slice name includes a unique identifier that corresponds to a particular copy number of the copies of the encoded data slices;

storing, by a set of storage units of the DSN, the plurality of sets of encoded data slices in accordance with the at least one set of write commands, wherein a storage unit of the set of storage units stores the number of copies of the encoded data slice in accordance with the write command;

receiving a plurality of retrieval requests from a plurality of requesting computing devices regarding the digital content; and

fulfilling, in accordance with a fulfillment protocol, the plurality of retrieval requests utilizing various combinations of copies of encoded data slices of the plurality of sets of encoded data slices.

2. The method of claim 1, wherein the encoding comprises:

dividing the digital content into a plurality of data segments; and

dispersed storage error encoding a data segment of the plurality of data segments to produce the set of encoded data slices, wherein a decode threshold number of encoded data slices of the set of encoded data slices is needed to recover the data segment.

3. The method of claim 1 further comprises:

generating the write command to further include a command to create the copies of the encoded data slices.

4. The method of claim 1 further comprises:

determining a first number of copies of encoded data slices for a first storage unit of the set of storage units based on data processing capabilities of the first storage unit; and

determining a second number of copies of encoded data slices for a second storage unit of the set of storage units based on data processing capabilities of the second storage unit.

5. The method of claim 1 further comprises:

storing, by the storage unit, a first copy of the copies of the encoded data slice in a first memory device; and

storing, by the storage unit, a second copy of the copies of the encoded data slice in a second memory device, wherein the first and second memory devices are independently accessible within the storage unit.

6. The method of claim 1 further comprises:

as the plurality of retrieval requests is received, assigning corresponding requesting computing devices of the plurality of requesting computing devices to copies of the plurality of sets of encoded data slices, wherein a first requesting computing device is assigned to a first copy of encoded data slices of the plurality of sets of encoded data slices.

7. The method of claim 1, wherein the fulfillment protocol comprises one of:

assigning groups of requesting computing devices to copies of the plurality of sets of encoded data slices, wherein a first group of requesting computing devices is assigned to a first copy of encoded data slices of the plurality of sets of encoded data slices; and

instructing storage units of the set of storage units as to which copy of copies of encoded data slices to output in response to a particular requesting computing device'"'"'s retrieval request.

8. The method of claim 1, wherein the fulfilling the plurality of retrieval requests comprises:

sending, by a first sub-set of storage units of the set of storage units, a first unique combination of a decode threshold number of encoded data slices of the plurality of sets of encoded data slices from one or more copies of the plurality of sets of encoded data slices to a first requesting computing device of the plurality of requesting computing devices; and

sending, by a second sub-set of storage units of the set of storage units, a second unique combination of the decode threshold number of encoded data slices of the plurality of sets of encoded data slices from another one or more copies of the plurality of sets of encoded data slices to a second requesting computing device of the plurality of requesting computing devices.

9. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

encode, by a first computing device of a dispersed storage network (DSN), digital content into a plurality of sets of encoded data slices;

generate, by the first computing device, at least one set of write commands regarding the plurality of sets of encoded data slices, wherein a write command of the at least one set of write commands is regarding an encoded data slice of a set of the plurality of sets of encoded data slices and includes an indication of a number of copies of the encoded data slice to be stored, wherein the write command is generated to include a set of slice names for the copies of the encoded data slices, and wherein each slice name includes a unique identifier that corresponds to a particular copy number of the copies of the encoded data slices;

store, by a set of storage units of the DSN, the plurality of sets of encoded data slices in accordance with the at least one set of write commands, wherein a storage unit of the set of storage units stores the number of copies of the encoded data slice in accordance with the write command;

receive a plurality of retrieval requests from a plurality of requesting computing devices regarding the digital content; and

fulfill, in accordance with a fulfillment protocol, the plurality of retrieval requests utilizing various combinations of copies of encoded data slices of the plurality of sets of encoded data slices.

10. The non-transitory computer readable storage medium of claim 9, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to encode by:

dividing the digital content into a plurality of data segments; and

dispersed storage error encoding a data segment of the plurality of data segments to produce the set of encoded data slices, wherein a decode threshold number of encoded data slices of the set of encoded data slices is needed to recover the data segment.

11. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

generate the write command to further include a command to create the copies of the encoded data slices.

12. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine a first number of copies of encoded data slices for a first storage unit of the set of storage units based on data processing capabilities of the first storage unit; and

determine a second number of copies of encoded data slices for a second storage unit of the set of storage units based on data processing capabilities of the second storage unit.

13. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

store, by the storage unit, a first copy of the copies of the encoded data slice in a first memory device; and

store, by the storage unit, a second copy of the copies of the encoded data slice in a second memory device, wherein the first and second memory devices are independently accessible within the storage unit.

14. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

as the plurality of retrieval requests is received, assign corresponding requesting computing devices of the plurality of requesting computing devices to copies of the plurality of sets of encoded data slices, wherein a first requesting computing device is assigned to a first copy of encoded data slices of the plurality of sets of encoded data slices.

15. The non-transitory computer readable storage medium of claim 9, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to execute the fulfillment protocol by one of:

assigning groups of requesting computing devices to copies of the plurality of sets of encoded data slices, wherein a first group of requesting computing devices is assigned to a first copy of encoded data slices of the plurality of sets of encoded data slices; and

instructing storage units of the set of storage units as to which copy of copies of encoded data slices to output in response to a particular requesting computing device'"'"'s retrieval request.

16. The non-transitory computer readable storage medium of claim 9, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to fulfill the plurality of retrieval requests by:

sending, by a first sub-set of storage units of the set of storage units, a first unique combination of a decode threshold number of encoded data slices of the plurality of sets of encoded data slices from one or more copies of the plurality of sets of encoded data slices to a first requesting computing device of the plurality of requesting computing devices; and

sending, by a second sub-set of storage units of the set of storage units, a second unique combination of the decode threshold number of encoded data slices of the plurality of sets of encoded data slices from another one or more copies of the plurality of sets of encoded data slices to a second requesting computing device of the plurality of requesting computing devices.