Utilizing local memory and dispersed storage memory to access encoded data slices

• US 8,843,803 B2
• Filed: 03/06/2012
• Issued: 09/23/2014
• Est. Priority Date: 04/01/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): Merant, Guerrier
• Assistant Examiner(s): GANDHI, DIPAKKUMAR B
• Application Number:
  US13/413,436
• Publication Number:
  US 20120254690A1
• Time in Patent Office: 931 Days
• US Class Current: 714/763
• CPC Class Codes: G06F 11/1044 with specific ECC/EDC distribution
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1441 Resetting or repowering
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0646 Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/09 Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/23 using convolutional codes, e.g. unit memory codes
• CPC Class Codes: H03M 13/2707 Simple row-column interleaver, i.e. pure block interleaving
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module encoding data utilizing a dispersed storage error coding function to produce a set of encoded data slices, wherein the dispersed storage error coding function includes a decode threshold parameter and a pillar width parameter. The method continues with the processing module storing a number of encoded data slices of the set of encoded data slices in a local memory, wherein the number is based on the decode threshold parameter and is less than the pillar width parameter, and outputting remaining encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory.

Claims

1. A method comprises:

encoding data utilizing a dispersed storage error coding function to produce a set of encoded data slices, wherein the dispersed storage error coding function includes a decode threshold parameter and a pillar width parameter;

storing a number of encoded data slices of the set of encoded data slices in a local memory, wherein the number is based on the decode threshold parameter and is less than the pillar width parameter; and

outputting remaining encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory.

2. The method of claim 1 further comprises:

determining the number of encoded data slices by at least one of;

performing a mathematical function on the decode threshold parameter; and

performing a second mathematical function based on at least one of physical structure of the local memory and performance characteristics of the local memory.

3. The method of claim 1 further comprises:

encoding the data utilizing the dispersed storage error coding function to produce a plurality of sets of encoded data slices, which includes the set of encoded data slices; and

determining, in accordance with a set storage protocol, the number of encoded data slices of one or more sets of the plurality of sets of encoded data slices.

4. The method of claim 1, wherein the encoding the data further comprises:

establishing the decode threshold parameter and the pillar width parameter based on one or more of physical characteristics of the local memory and a local memory performance characteristic.

5. The method of claim 1, wherein the storing the number of encoded data slices further comprises:

selecting the number of encoded data slices from the set of encoded data slices based on the dispersed storage error coding function; and

issuing a number of write requests to the local memory for the number of encoded data slices.

6. The method of claim 1, wherein the outputting remaining encoded data slices further comprises:

sending the remaining encoded data slices to a dispersed storage (DS) processing unit;

or

sending the data and the number to the DS processing unit, wherein the DS processing unit;

encodes the data utilizing the dispersed storage error coding function to produce another set of encoded data slices;

identifies the remaining encoded data slices from the other set of encoded data slices based on the number and on a set by set basis or for groupings of sets of encoded data slices; and

outputs the remaining encoded data slices to the DSN memory; and

updating an encoded data slice mapping.

7. A method comprises:

receiving a retrieval request for data, wherein the data is encoded utilizing a dispersed storage error coding function to produce a set of encoded data slices, wherein a number of encoded data slices of the set of encoded data slices are stored in a local memory and remaining encoded data slices of the set of encoded data slices are stored in dispersed storage network (DSN) memory;

issuing a number of data read requests to the local memory for retrieval of the number of encoded data slices;

determining whether a decode threshold number of encoded data slices have been received from the local memory;

when the decode threshold number of encoded data slices have not been received from the local memory, issuing one or more data read requests to the DSN memory for retrieving one or more of the remaining encoded data slices; and

when the decode threshold number of encoded data slices have been received, decoding the decode threshold number of encoded data slices using the dispersed storage error coding function to reproduce the data.

8. The method of claim 7, wherein the issuing the number of data read requests comprises:

determining the number of data read requests by at least one of;

performing a number look up operation;

performing a mathematical function on a decode threshold parameter; and

performing a second mathematical function based on at least one of physical structure of the local memory and performance characteristics of the local memory, wherein the dispersed storage error coding function includes a decode threshold parameter and a pillar width parameter.

9. The method of claim 7, wherein the determining whether the decode threshold number of encoded data slices have been received comprises one of:

determining that the decode threshold number of encoded data slices have not been received when the number of data read requests is less than a decode threshold parameter; and

when the number of data read requests is greater than or equal to the decode threshold parameter decode threshold, determining whether the decode threshold number of encoded data slices have been received within a given time frame.

10. The method of claim 7, wherein the issuing the one or more data read requests to the DSN memory further comprises:

selecting the one or more of the remaining encoded data slices based on one or more of;

an encoded data slice mapping retrieval;

a query;

a message; and

the data retrieval request.

11. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

encode data utilizing a dispersed storage error coding function to produce a set of encoded data slices, wherein the dispersed storage error coding function includes a decode threshold parameter and a pillar width parameter;

a second module, when operable within the computing device, causes the computing device to;

store a number of encoded data slices of the set of encoded data slices in a local memory, wherein the number is based on the decode threshold parameter and is less than the pillar width parameter; and

a third module, when operable within the computing device, causes the computing device to;

output remaining encoded data slices of the set of encoded data slices to dispersed storage network (DSN) memory.

12. The DS module of claim 11 further comprises:

the second module, when operable within the computing device, further causes the computing device to determine the number of encoded data slices by at least one of;

performing a mathematical function on the decode threshold parameter; and

performing a second mathematical function based on at least one of physical structure of the local memory and performance characteristics of the local memory.

13. The DS module of claim 11 further comprises:

the first module, when operable within the computing device, further causes the computing device to encode the data utilizing the dispersed storage error coding function to produce a plurality of sets of encoded data slices, which includes the set of encoded data slices; and

the second module, when operable within the computing device, further causes the computing device to determine, in accordance with a set storage protocol, the number of encoded data slices of one or more sets of the plurality of sets of encoded data slices.

14. The DS module of claim 11, wherein the first module further functions to encode the data by:

establishing the decode threshold parameter and the pillar width parameter based on one or more of physical characteristics of the local memory and a local memory performance characteristic.

15. The DS module of claim 11, wherein the second module further functions to store the number of encoded data slices by:

selecting the number of encoded data slices from the set of encoded data slices based on the dispersed storage error coding function; and

issuing a number of write requests to the local memory for the number of encoded data slices.

16. The DS module of claim 11, wherein the third module further functions to output remaining encoded data slices by:

sending the remaining encoded data slices to a dispersed storage (DS) processing unit;

or

sending the data and the number to the DS processing unit, wherein the DS processing unit;

encodes the data utilizing the dispersed storage error coding function to produce another set of encoded data slices;

identifies the remaining encoded data slices from the other set of encoded data slices based on the number and a set by set basis or for groupings of sets of encoded data slices; and

outputs the remaining encoded data slices to the DSN memory; and

updating an encoded data slice mapping.

17. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

receive a retrieval request for data, wherein the data is encoded utilizing a dispersed storage error coding function to produce a set of encoded data slices, wherein a number of encoded data slices of the set of encoded data slices are stored in a local memory and remaining encoded data slices of the set of encoded data slices are stored in dispersed storage network (DSN) memory;

a second module, when operable within the computing device, causes the computing device to;

issue a number of data read requests to the local memory for retrieval of the number of encoded data slices;

a third module, when operable within the computing device, causes the computing device to;

determine whether a decode threshold number of encoded data slices have been received from the local memory;

when the decode threshold number of encoded data slices have not been received from the local memory, a fourth module, when operable within the computing device, causes the computing device to issue one or more data read requests to the DSN memory for retrieving one or more of the remaining encoded data slices; and

when the decode threshold number of encoded data slices have been received, a fifth module, when operable within the computing device, causes the computing device to decode the decode threshold number of encoded data slices using the dispersed storage error coding function to reproduce the data.

18. The DS module of claim 17, wherein the second module functions to issue the number of data read requests by:

determining the number of data read requests by at least one of;

performing a number look up operation;

performing a mathematical function on a decode threshold parameter; and

performing a second mathematical function based on at least one of physical structure of the local memory and performance characteristics of the local memory, wherein the dispersed storage error coding function includes a decode threshold parameter and a pillar width parameter.

19. The DS module of claim 17, wherein the third module functions to determine whether a decode threshold number of encoded data slices have been received by one of:

determining that the decode threshold number of encoded data slices have not been received when the number of data read requests is less than a decode threshold parameter; and

when the number of data read requests is greater than or equal to the decode threshold parameter decode threshold, determining whether the decode threshold number of encoded data slices have been received within a given time frame.

20. The DS module of claim 17, wherein the fourth module further functions to issue the one or more data read requests to the DSN memory by:

selecting the one or more of the remaining encoded data slices based on one or more of;

an encoded data slice mapping retrieval;

a query;

a message; and

the data retrieval request.