Storing data using a dual path storage approach

• US 9,740,547 B2
• Filed: 12/01/2015
• Issued: 08/22/2017
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Gladwin, S. Christopher
• Primary Examiner(s): Leibovich, Yair
• Application Number:
  US14/955,200
• Publication Number:
  US 20160224403A1
• Time in Patent Office: 630 Days
• Field of Search: 714 624
• CPC Class Codes: G06F 11/0718 in an object-oriented system
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0739 in a data processing system embedded in automotive or aircraft systems

Abstract

A method begins by a processing module of a dispersed storage network (DSN) receiving a data object for storage in DSN memory and determining dispersed storage error encoding parameters for encoding the data object to produce a plurality of sets of encoded data slices. The method continues with the processing module determining to use a dual storage path approach for storing the plurality of sets of encoded data slices in first memory type of the DSN memory and in second memory type of the DSN memory. The method continues with the processing module dispersed storage error encoding the data object to produce the plurality of sets of encoded data slices and outputting the plurality of sets of encoded data slices to the first and second memory types of the DSN memory in accordance with the dispersed storage error encoding parameters.

Claims

1. A method for execution by a computing device of a dispersed storage network (DSN), the method comprises:

receiving a data object for storage in DSN memory;

determining dispersed storage error encoding parameters for encoding the data object into a plurality of sets of encoded data slices;

determining to use a dual storage path approach for storing the plurality of sets of encoded data slices in first memory type of the DSN memory and in second memory type of the DSN memory when a cumulative time difference between storing the plurality of sets of encoded data slices in the first memory type and the second memory type is equal to or greater than a storage time difference threshold;

dispersed storage error encoding the data object to produce the plurality of sets of encoded data slices;

outputting the plurality of sets of encoded data slices to the first memory type of the DSN memory in accordance with a first parameter set of the dispersed storage error encoding parameters; and

outputting the plurality of sets of encoded data slices to the second memory type of the DSN memory in accordance with a second parameter set of the dispersed storage error encoding parameters.

2. The method of claim 1 further comprises:

estimating a first storage time for storing the plurality of sets of encoded data slices in the first memory type of the DSN memory;

estimating a second storage time for storing the plurality of sets of encoded data slices in the second memory type of the DSN memory, wherein, on a set of encode data slices basis, the first memory type has a faster access time than the second memory type; and

when the estimated second storage time is equal to or greater than the estimated first storage time by the storage time difference threshold, determining to use the dual storage path approach for storing the plurality of encoded data slices in the first memory type and in the second memory type.

3. The method of claim 1 further comprises:

the first memory type being memory coupled to the computing device via a first communication link; and

the second memory type being memory coupled to the computing device via a second communication link, wherein the first communication link has a higher data rate than the second communication link.

4. The method of claim 1 further comprises:

the first memory type being memory having a first memory access rate; and

the second memory type being memory having a second memory access rate, wherein the first memory access rate is greater than the second memory access rate.

5. The method of claim 1 further comprises:

the first parameter set of the dispersed storage error encoding parameters including a decode threshold number, a set total number, and a first write threshold number, wherein the decode threshold number corresponds to a minimum number of encoded data slices of a set of encoded data slices needed to recover a corresponding data segment of the data object, the set total number corresponds to a total number of encoded data slices in the set of encoded data slices, and the first write threshold number corresponds to a first number of encoded data slices of the set of encoded data slices that needs to be successfully stored for the set of encoded data slices to be deemed successfully stored in the first memory type; and

the second parameter set of the dispersed storage error encoding parameters including the decode threshold number, the set total number, and a second write threshold number, wherein the second write threshold number corresponds to a second number of encoded data slices of the set of encoded data slices that needs to be successfully stored for the set of encoded data slices to be deemed successfully stored in the second memory type, wherein the second write threshold is a greater number than the first write threshold.

6. The method of claim 1 further comprises:

the first parameter set of the dispersed storage error encoding parameters including a decode threshold number, a first set total number, and a first write threshold number, wherein the decode threshold number corresponds to a minimum number of encoded data slices of a set of encoded data slices needed to recover a corresponding data segment of the data object, the first set total number corresponds to a first total number of encoded data slices in the set of encoded data slices, and the first write threshold number corresponds to a first number of encoded data slices of the set of encoded data slices that needs to be successfully stored for the set of encoded data slices to be deemed successfully stored in the first memory type; and

the second parameter set of the dispersed storage error encoding parameters including the decode threshold number, a second set total number, and a second write threshold number, wherein the second set total number corresponds to a second total number of encoded data slices in the set of encoded data slices and the second write threshold number corresponds to a second number of encoded data slices of the set of encoded data slices that needs to be successfully stored for the set of encoded data slices to be deemed successfully stored in the second memory type, wherein the second write threshold is a greater number than the first write threshold and the second set total number is a greater number than the first set total number.

7. The method of claim 1 further comprises:

when the plurality of sets of encoded data slices have been successfully stored in the second memory type, deleting encoded data slices from the first memory type such that, for each set of the plurality of sets of encoded data slices, the first memory type is storing a decode threshold number of encoded data slices.

8. The method of claim 1 further comprises:

generating a plurality of sets of slices names for the plurality of sets of encoded data slices;

generating a first plurality of DSN addresses to include the plurality of sets of slices names and first DSN identifiers of storage units of the first memory type;

generating a second plurality of DSN addresses to include the plurality of sets of slices names and second DSN identifiers of storage units of the second memory type;

outputting, in accordance with the first plurality of DSN addresses, the plurality of sets of encoded data slices to the first memory type of the DSN memory; and

outputting, in accordance with the second plurality of DSN addresses, the plurality of sets of encoded data slices to the second memory type of the DSN memory.

9. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

receive a data object for storage in DSN memory;

determine dispersed storage error encoding parameters for encoding the data object into a plurality of sets of encoded data slices;

determine to use a dual storage path approach for storing the plurality of sets of encoded data slices in first memory type of the DSN memory and in second memory type of the DSN memory when a cumulative time difference between storing the plurality of sets of encoded data slices in the first memory type and the second memory type is equal to or greater than a storage time difference threshold;

dispersed storage error encode the data object to produce the plurality of sets of encoded data slices;

output the plurality of sets of encoded data slices to the first memory type of the DSN memory in accordance with a first parameter set of the dispersed storage error encoding parameters; and

output the plurality of sets of encoded data slices to the second memory type of the DSN memory in accordance with a second parameter set of the dispersed storage error encoding parameters.

10. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

estimate a first storage time for storing the plurality of sets of encoded data slices in the first memory type of the DSN memory;

estimate a second storage time for storing the plurality of sets of encoded data slices in the second memory type of the DSN memory, wherein, on a set of encode data slices basis, the first memory type has a faster access time than the second memory type; and

when the estimated second storage time is equal to or greater than the estimated first storage time by the storage time difference threshold, determine to use the dual storage path approach for storing the plurality of encoded data slices in the first memory type and in the second memory type.

11. The non-transitory computer readable storage medium of claim 9 further comprises:

the first memory type being memory coupled to the one or more computing devices via a first communication link; and

the second memory type being memory coupled to the one or more computing devices via a second communication link, wherein the first communication link has a higher data rate than the second communication link.

12. The non-transitory computer readable storage medium of claim 9 further comprises:

the first memory type being memory having a first memory access rate; and

the second memory type being memory having a second memory access rate, wherein the first memory access rate is greater than the second memory access rate.

13. The non-transitory computer readable storage medium of claim 9 further comprises:

the first parameter set of the dispersed storage error encoding parameters including a decode threshold number, a set total number, and a first write threshold number, wherein the decode threshold number corresponds to a minimum number of encoded data slices of a set of encoded data slices needed to recover a corresponding data segment of the data object, the set total number corresponds to a total number of encoded data slices in the set of encoded data slices, and the first write threshold number corresponds to a first number of encoded data slices of the set of encoded data slices that needs to be successfully stored for the set of encoded data slices to be deemed successfully stored in the first memory type; and

the second parameter set of the dispersed storage error encoding parameters including the decode threshold number, the set total number, and a second write threshold number, wherein the second write threshold number corresponds to a second number of encoded data slices of the set of encoded data slices that needs to be successfully stored for the set of encoded data slices to be deemed successfully stored in the second memory type, wherein the second write threshold is a greater number than the first write threshold.

14. The non-transitory computer readable storage medium of claim 9 further comprises:

the first parameter set of the dispersed storage error encoding parameters including a decode threshold number, a first set total number, and a first write threshold number, wherein the decode threshold number corresponds to a minimum number of encoded data slices of a set of encoded data slices needed to recover a corresponding data segment of the data object, the first set total number corresponds to a first total number of encoded data slices in the set of encoded data slices, and the first write threshold number corresponds to a first number of encoded data slices of the set of encoded data slices that needs to be successfully stored for the set of encoded data slices to be deemed successfully stored in the first memory type; and

the second parameter set of the dispersed storage error encoding parameters including the decode threshold number, a second set total number, and a second write threshold number, wherein the second set total number corresponds to a second total number of encoded data slices in the set of encoded data slices and the second write threshold number corresponds to a second number of encoded data slices of the set of encoded data slices that needs to be successfully stored for the set of encoded data slices to be deemed successfully stored in the second memory type, wherein the second write threshold is a greater number than the first write threshold and the second set total number is a greater number than the first set total number.

15. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

when the plurality of sets of encoded data slices have been successfully stored in the second memory type, delete encoded data slices from the first memory type such that, for each set of the plurality of sets of encoded data slices, the first memory type is storing a decode threshold number of encoded data slices.

16. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

generate a plurality of sets of slices names for the plurality of sets of encoded data slices;

generate a first plurality of DSN addresses to include the plurality of sets of slices names and first DSN identifiers of storage units of the first memory type;

generate a second plurality of DSN addresses to include the plurality of sets of slices names and second DSN identifiers of storage units of the second memory type;

output, in accordance with the first plurality of DSN addresses, the plurality of sets of encoded data slices to the first memory type of the DSN memory; and

output, in accordance with the second plurality of DSN addresses, the plurality of sets of encoded data slices to the second memory type of the DSN memory.