Transferring data of a dispersed storage network

• US 9,590,838 B2
• Filed: 09/17/2013
• Issued: 03/07/2017
• Est. Priority Date: 11/09/2010
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Motwani, Manish, Resch, Jason K., Baptist, Andrew, Volvovski, Ilya
• Primary Examiner(s): Thai, Tuan
• Assistant Examiner(s): Sain, Gautam
• Application Number:
  US14/029,347
• Publication Number:
  US 20140019579A1
• Time in Patent Office: 1,267 Days
• Field of Search: 711/165
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 12/02 Addressing or allocation; Relocation program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00
• CPC Class Codes: G06F 2211/1057 Parity-multiple bits-RAID6, i.e. RAID 6 implementations
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A method begins by a dispersed storage (DS) processing detecting unavailability of a storage device of a site of dispersed storage network (DSN) memory to produce an unavailable storage device. The method continues with the DS processing module reassigning a fraction of a logical address sub-range of the unavailable storage device to one or more other storage devices, rebuilding one or more logically addressable data objects to produce one or more rebuilt data objects and storing the one or more rebuilt data objects in the one or more other storage devices. When the unavailable storage device becomes available, the method continues with the DS processing module reallocating the fraction of the logical address sub-range from the one or more other storage devices to the storage device and transferring the one or more rebuilt data objects from the one or more other storage devices to the storage device.

Claims

1. A method for execution by one or more processing modules of a computing device of a dispersed storage network (DSN), the method comprises:

detecting unavailability of a storage device of a plurality of storage devices of a first site of a plurality of sites of DSN memory to produce an unavailable storage device, wherein the storage device includes a plurality of memory devices,

wherein each of a plurality of data objects is divided into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed storage error encoded into a set of encoded data slices having a set of logical DSN addresses, wherein the plurality of storage devices of the first site stores encoded data slices of the plurality of data segments of the plurality of data objects having logical DSN addresses that include first slice index information, and wherein the plurality of storage devices of a second site of the plurality of sites stores encoded data slices of the plurality of data segments of the plurality of data objects having logical DSN addresses that include second slice index information;

for encoded data slices stored by the unavailable storage device;

reassigning, within the first site, the logical DSN addresses of the encoded data slices stored by the unavailable storage device to one or more other storage devices of the plurality of storage devices;

rebuilding the encoded data slices stored by the unavailable storage device to produce rebuilt encoded data slices;

storing the rebuilt encoded data slices in the one or more other storage devices;

when the unavailable storage device becomes available, reallocating the logical DSN addresses of the encoded data slices stored by the unavailable storage device from the one or more other storage devices to the storage device; and

transferring, within the first site, the rebuilt encoded data slices objects from the one or more other storage devices to the storage device.

2. The method of claim 1, wherein the detecting the unavailability of the storage device comprises one of:

detecting unavailability of one or more memory devices of the plurality of memory devices associated with the storage device; and

detecting that the storage device is off-line.

3. The method of claim 1, wherein the reassigning the logical DSN addresses of the encoded data slices stored by the unavailable storage device comprises:

dividing the logical DSN addresses of the encoded data slices stored by the unavailable storage device into a first portion and a second portion, wherein a logical DSN address further includes data object identifying information and data segment identifying information;

reassigning the first portion to one of the one or more other storage devices having a first contiguous range of data object identifying information and data segment identifying information; and

reassigning the second portion to a second one of the one or more other storage devices having a second contiguous range of data object identifying information and data segment identifying information.

4. The method of claim 1, wherein the rebuilding the encoded data slices stored by the unavailable storage device comprises:

sending rebuilding requests to storage units of other sites regarding the encoded data slices stored by the unavailable storage device;

receiving, in response to the rebuilding requests, rebuilding data; and

rebuilding the encoded data slices stored by the unavailable storage device from the rebuilding data.

5. The method of claim 1, wherein the storing the rebuilt encoded data slices in the one or more other storage devices comprises:

sending write requests to the one or more other storage devices regarding the rebuilt encoded data slices, wherein a write request of the write requests includes one of the logical DSN addresses of the encoded data slices stored by the unavailable storage device and a corresponding one of the rebuilt encoded data slices.

6. The method of claim 1 further comprises:

detecting that the unavailable storage device is available based on one or more of;

a repair or replacement of the unavailable storage device;

the unavailable storage device coming back on-line; and

a repair or replacement of a memory device within the storage device.

7. The method of claim 1, wherein the transferring the rebuilt encoded data slices comprises:

issuing one or more write commands to the one or more other storage devices to write the rebuilt encoded data slices to the storage device;

receiving an acknowledge of successful storage of the rebuilt encoded data slices by the storage device; and

issuing a delete command to the one or more other storage devices to delete the rebuilt encoded data slices.

8. A dispersed storage (DS) processing module of a dispersed storage network (DSN), the DS processing module comprises:

a first module, when operable within a computing device, causes the computing device to;

detect unavailability of a storage device of a plurality of storage devices of a first site of a plurality of sites of DSN memory to produce an unavailable storage device, wherein the storage device includes a plurality of memory devices,

wherein each of a plurality of data objects is divided into a plurality of data segments,

wherein a data segment of the plurality of data segments is dispersed storage error encoded into a set of encoded data slices having a set of logical DSN addresses, wherein the plurality of storage devices of the first site stores encoded data slices of the plurality of data segments of the plurality of data objects having logical DSN addresses that include first slice index information, and wherein the plurality of storage devices of a second site of the plurality of sites stores encoded data slices of the plurality of data segments of the plurality of data objects having logical DSN addresses that include second slice index information; and

a second module, when operable within the computing device, causes the computing device to;

for encoded data slices stored by the unavailable storage device;

reassign, within the first site, the logical DSN addresses of the encoded data slices stored by the unavailable storage device to one or more other storage devices of the plurality of storage devices;

rebuild the encoded data slices stored by the unavailable storage device to produce rebuilt encoded data slices;

store the rebuilt encoded data slices in the one or more other storage devices;

when the unavailable storage device becomes available, reallocate the logical DSN addresses of the encoded data slices stored by the unavailable storage device from the one or more other storage devices to the storage device; and

transfer, within the first site, the rebuilt encoded data slices objects from the one or more other storage devices to the storage device.

9. The DS processing module of claim 8, wherein the first module functions to detect the unavailability of the storage device by one of:

detecting unavailability of one or more memory devices of the plurality of memory devices associated with the storage device; and

detecting that the storage device is off-line.

10. The DS processing module of claim 8, wherein the second module functions to reassign the logical DSN addresses of the encoded data slices stored by the unavailable storage device by:

dividing the logical DSN addresses of the encoded data slices stored by the unavailable storage device into a first portion and a second portion, wherein a logical DSN address further includes data object identifying information and data segment identifying information;

reassigning the first portion to one of the one or more other storage devices having a first contiguous range of data object identifying information and data segment identifying information; and

reassigning the second portion to a second one of the one or more other storage devices having a second contiguous range of data object identifying information and data segment identifying information.

11. The DS processing module of claim 8, wherein the second module functions to rebuild the encoded data slices stored by the unavailable storage device by:

sending rebuilding requests to storage units of other sites regarding the encoded data slices stored by the unavailable storage device;

receiving, in response to the rebuilding requests, rebuilding data; and

rebuilding the encoded data slices stored by the unavailable storage device from the rebuilding data.

12. The DS processing module of claim 8, wherein the second module functions to store the rebuilt encoded data slices in the one or more other storage devices by:

sending write requests to the one or more other storage devices regarding the rebuilt encoded data slices, wherein a write request of the write requests includes one of the logical DSN addresses of the encoded data slices stored by the unavailable storage device and a corresponding one of the rebuilt encoded data slices.

13. The DS processing module of claim 8 further comprises:

the second module, when operable within the computing device, further causes the computing device to;

detect that the unavailable storage device is available based on one or more of;

a repair or replacement of the unavailable storage device;

the unavailable storage device coming back on-line; and

a repair or replacement of a memory device within the storage device.

14. The DS processing module of claim 8, wherein the second module functions to transfer the rebuilt encoded data slices by:

issuing one or more write commands to the one or more other storage devices to write the rebuilt encoded data slices to the storage device;

receiving an acknowledge of successful storage of the rebuilt encoded data slices by the storage device; and

issuing a delete command to the one or more other storage devices to delete the rebuilt encoded data slices.