METHOD AND APPARATUS FOR DISPERSED STORAGE DATA TRANSFER

• US 20110078493A1
• Filed: 05/12/2010
• Published: 03/31/2011
• Est. Priority Date: 09/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: GRUBE, GARY W., MARKISON, TIMOTHY W.
• Granted Patent: US 9,448,730 B2
• Time in Patent Office: Days
• US Class Current: 714/6.3
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2053 where persistent mass storage functionality or persistent mass storage control functionality is redundant error detection or correction in information storage based on relative movement between record carrier and transducer G11B20/18
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/068 Hybrid storage device
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

The method begins with a processing module determining whether to reconstruct data corresponding to a plurality of data slices when the plurality of data slices is to be transferred from a first type of memory device to a second type of memory device. The method continues with the processing module retrieving the plurality of data slices from a first set of memory devices that are of the first type of memory, reconstructing at least a portion of the data from the plurality of data slice in accordance with a first error coding dispersal function to produce reconstructed data, encoding the reconstructed data in accordance with a second error coding dispersal function to produce a second plurality of data slices, and storing the second plurality of data slices in a second set of memory devices that are of the second type of memory when the data is to be reconstructed.

Claims

1. A method for dispersed memory storage comprises:

determining whether to transfer a plurality of data slices from a first type of memory device to a second type of memory device;

when the plurality of data slices is to be transferred from the first type of memory device to the second type of memory device, determining whether to reconstruct data corresponding to the plurality of data slices;

when the data is to be reconstructed;

retrieving the plurality of data slices from a first set of memory devices that are of the first type of memory;

reconstructing at least a portion of the data from the plurality of data slices in accordance with a first error coding dispersal function to produce reconstructed data;

encoding the reconstructed data in accordance with a second error coding dispersal function to produce a second plurality of data slices; and

storing the second plurality of data slices in a second set of memory devices that are of the second type of memory.

2. The method of claim 1, wherein the determining whether to transfer the plurality of data slices further comprises:

determining whether to transfer a plurality of data slices based on at least one of;

elapsed time period of storage in the first type of memory;

elapsed time period since a data slice retrieval from the first type of memory;

a first type of memory utilization indicator;

receiving a command; and

receiving a request.

3. The method of claim 1 further comprises:

when the data is not to be reconstructed;

retrieving the plurality of data slices from the first set of memory devices that are of the first type of memory;

storing the plurality of data slices in the second set of memory devices that are of the second type of memory; and

facilitating deletion of the plurality of data slices from the first set of memory devices that are of the first type of memory.

4. The method of claim 1, wherein the first error coding dispersal function comprises:

an error coding type that includes at least one of an error coding algorithm, an encryption algorithm, and a compression algorithm; and

operational parameters that include two or more of a pillar width, a read threshold, a slicing parameter, an integrity check method, a caching settings, and a parallelism settings.

5. The method of claim 1, wherein the second error coding dispersal function comprises:

an error coding type that includes at least one of an error coding algorithm, an encryption algorithm, and a compression algorithm; and

operational parameters that include two or more of a pillar width, a read threshold, a slicing parameter, an integrity check method, a caching settings, and a parallelism settings.

6. The method of claim 1 further comprises:

when the data is to be reconstructed;

retrieving pluralities of data slices from the first set of memory devices, wherein each of the pluralities of data slices corresponds to a data segment;

reconstructing the data from the pluralities of data slices in accordance with a first error coding dispersal function to produce reconstructed data;

encoding the reconstructed data in accordance with a second error coding dispersal function to produce a second pluralities of data slices; and

storing the second pluralities of data slices in the second set of memory devices.

7. The method of claim 1, wherein the at least a portion of the data comprises:

a data partition, wherein the data partition corresponds to at least one of;

customized data content; and

generic data content.

8. A computer comprises:

a dispersal memory interface; and

a processing module operable to;

determine whether to transfer a plurality of data slices from a first type of memory device to a second type of memory device;

when transferring the plurality of data slices from the first type of memory device to the second type of memory device, determine whether to reconstruct data corresponding to the plurality of data slices;

when the data is to be reconstructed;

retrieve, via the dispersal memory interface, the plurality of data slices from a first set of memory devices that are of the first type of memory;

reconstruct at least a portion of the data from the plurality of data slices in accordance with a first error coding dispersal function to produce reconstructed data;

encode the reconstructed data in accordance with a second error coding dispersal function to produce a second plurality of data slices; and

output, via the dispersal memory interface, the second plurality of data slices to a second set of memory devices that are of the second type of memory.

9. The computer of claim 8 further comprises at least one of:

a main memory;

a local non-main memory; and

a non-local non-main memory.

10. The computer of claim 8, wherein the dispersal memory interface may comprise at least one of:

a multi-general purpose input output; and

a plurality of interfaces.

11. The computer of claim 8, wherein the processing module determines whether to transfer a plurality of data slices by:

determining whether to transfer a plurality of data slices based on at least one of;

elapsed time period of storage in the first type of memory;

elapsed time period since a data slice retrieval, via the dispersal memory interface, from the first type of memory;

a first type of memory utilization indicator;

obtaining a command; and

obtaining a request.

12. The computer of claim 8, wherein the processing module further functions to:

when the data is not to be reconstructed;

retrieving, via the dispersal memory interface, the plurality of data slices from the first set of memory devices that are of the first type of memory;

outputting, via the dispersal memory interface, the plurality of data slices in the second set of memory devices that are of the second type of memory; and

facilitating deletion, via the dispersal memory interface, of the plurality of data slices from the first set of memory devices that are of the first type of memory.

13. The computer of claim 8, wherein the first error coding dispersal function comprises:

an error coding type that includes at least one of an error coding algorithm, an encryption algorithm, and a compression algorithm; and

operational parameters that include two or more of a pillar width, a read threshold, a slicing parameter, an integrity check method, a caching settings, and a parallelism settings.

14. The computer of claim 8, wherein the second error coding dispersal function comprises:

an error coding type that includes at least one of an error coding algorithm, an encryption algorithm, and a compression algorithm; and

operational parameters that include two or more of a pillar width, a read threshold, a slicing parameter, an integrity check method, a caching settings, and a parallelism settings.

15. The computer of claim 8, wherein the processing module further functions to:

when the data is to be reconstructed;

retrieving, via the dispersal memory interface, pluralities of data slices from the first set of memory devices, wherein each of the pluralities of data slices corresponds to a data segment;

reconstructing the data from the pluralities of data slices in accordance with a first error coding dispersal function to produce reconstructed data;

encoding the reconstructed data in accordance with a second error coding dispersal function to produce a second pluralities of data slices; and

outputting, via the dispersal memory interface, the second pluralities of data slices to the second set of memory devices.

16. The computer of claim 8, wherein the at least a portion of the data comprises:

a data partition, wherein the data partition corresponds to at least one of;

customized data content; and

generic data content.