Method and apparatus for selecting storage units to store dispersed storage data

• US 9,823,861 B2
• Filed: 05/30/2014
• Issued: 11/21/2017
• Est. Priority Date: 09/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): SCHELL, JOSEPH O
• Application Number:
  US14/292,585
• Publication Number:
  US 20140281225A1
• Time in Patent Office: 1,271 Days
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2053 where persistent mass storage functionality or persistent mass storage control functionality is redundant error detection or correction in information storage based on relative movement between record carrier and transducer G11B20/18
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/068 Hybrid storage device
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

The method begins with a processing module determining whether to reconstruct data corresponding to a plurality of data slices when the plurality of data slices is to be transferred from a first type of memory device to a second type of memory device. The method continues with the processing module retrieving the plurality of data slices from a first set of memory devices that are of the first type of memory, reconstructing at least a portion of the data from the plurality of data slice in accordance with a first error coding dispersal function to produce reconstructed data, encoding the reconstructed data in accordance with a second error coding dispersal function to produce a second plurality of data slices, and storing the second plurality of data slices in a second set of memory devices that are of the second type of memory when the data is to be reconstructed.

Claims

1. A method comprising:

encoding, by a processing module of a computing device, a data segment of a data object into a set of encoded data slices;

determining, by the processing module, storage requirements of the data object;

determining, by the processing module, memory device capabilities of a plurality of distributed storage units based on types of memory devices, wherein at least one of the distributed storage units of the plurality of distributed storage units includes multiple types of memory devices, and wherein a first type of memory device has first memory characteristics and a second type of memory device has second memory characteristics;

determining, by the processing module, a storage mode based on one or more of the storage requirements of the data object, the memory device capabilities of a dispersed storage network (DSN) memory, and a type of data, the storage mode including a time phase indicator specifying one or more time intervals for a given set of storage requirements;

identifying, by the processing module, a set of distributed storage units of the plurality of distributed storage units that have at least one or more of the multiple types of memory devices based on the storage mode; and

sending, by the computing device, at least a write threshold number of encoded data slices of the data segment to the set of distributed storage units for storage in the at least one or more of the multiple types of memory devices in accordance with the storage mode, wherein the write threshold number is greater than a decode threshold number and less than a total number, wherein the decode threshold number corresponds to a minimum number of encoded data slices of the set of encoded data slices that is needed to recover the data segment, wherein the total number corresponds to a number of encoded data slices in the set of encoded data slices.

2. The method of claim 1 further comprises:

the storage requirements including one or more of;

a file type, file size, priority, security index, estimated storage time, and estimated time between retrievals;

the memory device capabilities including one or more of;

a memory device storage cost, a memory device storage access speed, a memory device storage reliability, a memory device storage availability, and a memory device storage capacity; and

the memory device characteristics including one or more of;

speed of access, cost, reliability, availability, capacity and memory parameters.

3. The method of claim 1 further comprises:

the set of distributed storage units includes a write threshold number of distributed storage units having a particular type of memory device and a redundancy number of distributed storage units having another type of memory device.

4. The method of claim 1 further comprising determining the storage requirements based on storage metadata of the data object.

5. The method of claim 1 further comprises:

determining, by the processing module, to move the write threshold number of encoded data slices from a particular type of memory device to another type of memory device within the plurality of distributed storage units; and

transferring, by the computing device, the write threshold number of encoded data slices from the particular type of memory device to the other type of memory device.

6. The method of claim 5, wherein the transferring comprises:

retrieving, by the processing module, the write threshold number of encoded data slices;

recovering, by the processing module, the data segment from the write threshold number of encoded data slices;

re-encoding, by the processing module, the data segment to produce a second set of encoded data slices; and

sending, by the computing device, a write threshold number of encoded data slices of the second set of encoded data slices.

7. A computing device comprising:

an interface to receive a data segment and to communicate with a plurality of distributed storage units; and

a processing module operable to;

encode a data segment of a data object into a set of encoded data slices;

determine storage requirements of the data object;

determine memory device capabilities of a plurality of distributed storage units based on types of memory devices, wherein at least one of the distributed storage units of the plurality of distributed storage units includes multiple types of memory devices, and wherein a first type of memory device has first memory characteristics and a second type of memory device has second memory characteristics;

determine, by the processing module, a storage mode based on one or more of the storage requirements of the data object, the memory device capabilities of a dispersed storage network (DSN) memory, and a type of data, the storage mode including a time phase indicator specifying one or more time intervals for a given set of storage requirements;

identify a set of distributed storage units of the plurality of distributed storage units that have at least one or more of the multiple types of memory devices based on the storage mode; and

send, via the interface, at least a write threshold number of encoded data slices of the data segment to the set of distributed storage units for storage in the at least one or more of the multiple types of memory devices in accordance with the storage mode, wherein the write threshold number is greater than a decode threshold number and less than a total number, wherein the decode threshold number corresponds to a minimum number of encoded data slices of the set of encoded data slices that is needed to recover the data segment, wherein the total number corresponds to a number of encoded data slices in the set of encoded data slices.

8. The computing device of claim 7 further comprises:

the storage requirements including one or more of;

a file type, file size, priority, security index, estimated storage time, and estimated time between retrievals;

the memory device capabilities including one or more of;

a memory device storage cost, a memory device storage access speed, a memory device storage reliability, a memory device storage availability, and a memory device storage capacity; and

the memory device characteristics including one or more of;

speed of access, cost, reliability, availability, capacity and memory parameters.

9. The computing device of claim 7 further comprises:

the set of distributed storage units includes a write threshold number of distributed storage units having a particular type of memory device and a redundancy number of distributed storage units having another type of memory device.

10. The computing device of claim 7, wherein the processing module, is further operable to determine the storage requirements based on storage metadata of the data object.

11. The computing device of claim 7, wherein the processing module, is further operable to:

determine to move the write threshold number of encoded data slices from a particular type of memory device to another type of memory device within the plurality of distributed storage units; and

transfer the write threshold number of encoded data slices from the particular type of memory device to the other type of memory device.

12. The computing device of claim 11, wherein the processing module, is further operable to transfer the write threshold number of encoded data slices by:

retrieving the write threshold number of encoded data slices;

recovering the data segment from the write threshold number of encoded data slices;

re-encoding the data segment to produce a second set of encoded data slices; and

sending, via the interface, a write threshold number of encoded data slices of the second set of encoded data slices.