Utilizing local memory and dispersed storage memory to access encoded data slices

• US 9,110,834 B2
• Filed: 09/09/2014
• Issued: 08/18/2015
• Est. Priority Date: 04/01/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): Blair, April Y
• Assistant Examiner(s): GANDHI, DIPAKKUMAR B
• Application Number:
  US14/480,945
• Publication Number:
  US 20150012798A1
• Time in Patent Office: 343 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1044 with specific ECC/EDC distribution
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1441 Resetting or repowering
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0646 Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/09 Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/23 using convolutional codes, e.g. unit memory codes
• CPC Class Codes: H03M 13/2707 Simple row-column interleaver, i.e. pure block interleaving
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module establishing a dispersed storage (DS) error coding function based on a number of local memories associated with the computing device, wherein a decode threshold number of the DS error coding function corresponds to the number of local memories. The method continues with the processing module encoding a data segment of data utilizing the DS error coding function to produce a set of encoded data slices, of which the decode threshold number of encoded data slices is required to recover the data and the set of encoded data slices includes a pillar width number of encoded data slices. The method continues with the processing module storing the decode threshold number of encoded data slices in the local memories and outputting a remaining number of encoded data slices of the set of encoded data slices to the dispersed storage network (DSN) memory for storage therein.

Claims

1. A method for execution by a computing device, the method comprises:

establishing a dispersed storage error coding function based on a number of local memories associated with the computing device, wherein a decode threshold number of the dispersed storage error coding function corresponds to the number of local memories;

encoding a data segment of data utilizing the dispersed storage error coding function to produce a set of encoded data slices, wherein the decode threshold number of encoded data slices of the set of encoded data slices are required to recover the data and wherein the set of encoded data slices includes a pillar width number of encoded data slices;

storing the decode threshold number of encoded data slices in the local memories; and

outputting a remaining number of encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory for storage therein, wherein the remaining number is based on a difference between the pillar width number and the decode threshold number.

2. The method of claim 1 further comprises:

determining the number of local memories based on one or more of;

internal memory devices within the computing device;

determining a number of the internal memory devices that are available for storing encoded data slices;

external memory devices available to the computing device with a local area network; and

determining a number of the external memory devices that are available for storing encoded data slices.

3. The method of claim 1, wherein the outputting the remaining number of encoded data slices further comprises:

sending the remaining number of encoded data slices to a dispersed storage (DS) processing unit;

or

sending the data segment and the remaining number of encoded data slices, wherein the DS processing unit;

encodes the data segment utilizing the dispersed storage error coding function to produce another set of encoded data slices;

identifies the remaining number of encoded data slices from the other set of encoded data slices based on the remaining number; and

outputs the remaining number of encoded data slices to the DSN memory; and

updating an encoded data slice mapping.

4. The method of claim 1, wherein the storing the decode threshold number of encoded data slices comprises:

issuing a number of write requests to the local memories for the decode threshold number of encoded data slices.

5. The method of claim 1 further comprises:

encoding another data segment of data utilizing the dispersed storage error coding function to produce another set of encoded data slices;

storing the decode threshold number of the other encoded data slices in the local memories; and

outputting the remaining number of encoded data slices of the other set of encoded data slices to dispersed storage network (DSN) memory for storage therein.

6. A computing device comprises:

an interface;

memory;

a processing module operably coupled to the interface and the memory, wherein the processing module is operable to;

establish a dispersed storage error coding function based on a number of local memories associated with the computing device, wherein a decode threshold number of the dispersed storage error coding function corresponds to the number of local memories;

encode a data segment of data utilizing the dispersed storage error coding function to produce a set of encoded data slices, wherein the decode threshold number of encoded data slices of the set of encoded data slices are required to recover the data and wherein the set of encoded data slices includes a pillar width number of encoded data slices;

store the decode threshold number of encoded data slices in the local memories; and

output a remaining number of encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory for storage therein, wherein the remaining number is based on a difference between the pillar width number and the decode threshold number.

7. The computing device of claim 6 wherein the processing module is further operable to:

determine the number of local memories based on one or more of;

internal memory devices within the computing device;

determining a number of the internal memory devices that are available for storing encoded data slices;

external memory devices available to the computing device with a local area network; and

determining a number of the external memory devices that are available for storing encoded data slices.

8. The computing device of claim 6, wherein the processing module in the outputting the remaining number of encoded data slices further functions to:

send the remaining number of encoded data slices to a dispersed storage (DS) processing unit;

or

send the data segment and the remaining number of encoded data slices, wherein the DS processing unit;

encodes the data segment utilizing the dispersed storage error coding function to produce another set of encoded data slices;

identifies the remaining number of encoded data slices from the other set of encoded data slices based on the remaining number; and

outputs the remaining number of encoded data slices to the DSN memory; and

update an encoded data slice mapping.

9. The computing device of claim 6, wherein the processing module in the storing the decode threshold number of encoded data slices further functions to:

issue a number of write requests to the local memories for the decode threshold number of encoded data slices.

10. The computing device of claim 6, wherein the processing module further functions to:

encode another data segment of data utilizing the dispersed storage error coding function to produce another set of encoded data slices;

store the other decode threshold number of the other encoded data slices in the local memories; and

output another remaining number of encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory for storage therein, wherein the other remaining number is based on a difference between the pillar width number and the decode threshold number.

11. A non-transitory computer readable storage device comprises:

a first memory section storing operational instructions that, when executed by a computing device, causes the computing device to;

establish a dispersed storage error coding function based on a number of local memories associated with the computing device, wherein a decode threshold number of the dispersed storage error coding function corresponds to the number of local memories;

a second memory section storing operational instructions that, when executed by the computing device, causes the computing device to;

encode a data segment of data utilizing the dispersed storage error coding function to produce a set of encoded data slices, wherein the decode threshold number of encoded data slices of the set of encoded data slices are required to recover the data and wherein the set of encoded data slices includes a pillar width number of encoded data slices;

a third memory section storing operational instructions that, when executed by the computing device, causes the computing device to;

store the decode threshold number of encoded data slices in the local memories; and

a fourth memory section storing operational instructions that, when executed by the computing device, causes the computing device to;

output a remaining number of encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory for storage therein, wherein the remaining number is based on a difference between the pillar width number and the decode threshold number.

12. The non-transitory computer readable storage device of claim 11, wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:

determine the number of local memories based on one or more of;

internal memory devices within the computing device;

determining a number of the internal memory devices that are available for storing encoded data slices;

external memory devices available to the computing device with a local area network; and

determining a number of the external memory devices that are available for storing encoded data slices.

13. The non-transitory computer readable storage device of claim 11 further comprises:

the fourth memory section stores further operational instructions that, when executed by the computing device, causes the computing device to;

send the remaining number of encoded data slices to a dispersed storage (DS) processing unit;

or

send the data segment and the remaining number of encoded data slices to the DS processing unit;

a fifth memory section that stores operational instructions that, when executed by the DS processing unit, causes the DS processing unit to;

encode the data segment utilizing the dispersed storage error coding function to produce another set of encoded data slices;

identify the remaining number of encoded data slices from the other set of encoded data slices based on the remaining number; and

output the remaining number of encoded data slices to the DSN memory; and

a sixth memory section stores further operational instructions that, when executed by the computing device, causes the computing device to;

update an encoded data slice mapping.

14. The non-transitory computer readable storage device of claim 11 wherein the third memory section stores further operational instructions that, when executed by the computing device, causes the computing device to:

issue a number of write requests to the local memories for the decode threshold number of encoded data slices.

15. The non-transitory computer readable storage device of claim 11 further comprises:

the second memory section stores further operational instructions that, when executed by the computing device, causes the computing device to encode another data segment of data utilizing the dispersed storage error coding function to produce another set of encoded data slices;

the third memory section stores further operational instructions that, when executed by the computing device, causes the computing device to store the other decode threshold number of the other encoded data slices in the local memories; and

the fourth memory section stores further operational instructions that, when executed by the computing device, causes the computing device to output another remaining number of encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory for storage therein, wherein the other remaining number is based on a difference between the pillar width number and the decode threshold number.