DATA MIGRATION IN A DISPERSED STORAGE NETWORK

• US 20110185141A1
• Filed: 11/10/2010
• Published: 07/28/2011
• Est. Priority Date: 01/28/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: DHUSE, GREG, Volvovski, Ilya, Resch, Jason K., Leggette, Wesley, Baptist, Andrew
• Granted Patent: US 8,954,667 B2
• Time in Patent Office: Days
• US Class Current: 711/165
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by processing module determining data to migrate, wherein the data is stored as a plurality of sets of encoded data slices in a first set of dispersed storage (DS) units. The method continues with the processing module retrieving at least a read threshold number of encoded data slices for each set of the plurality of sets of encoded data slices and dispersed storage error decoding the at least the read threshold number of encoded data slices in accordance with error coding dispersal storage function parameters to reproduce the data. The method continues with the processing module dispersed storage error encoding the data in accordance with second error coding dispersal storage function parameters to produce a plurality of sets of second encoded data slices and sending at least a write threshold number of second encoded data slices to a second set of DS units for storage therein.

Claims

1. A method comprises:

determining data to migrate, wherein the data is stored as a plurality of sets of encoded data slices in a first set of dispersed storage (DS) units;

retrieving at least a read threshold number of encoded data slices for each set of the plurality of sets of encoded data slices;

dispersed storage error decoding the at least the read threshold number of encoded data slices for each set of the plurality of sets of encoded data slices in accordance with error coding dispersal storage function parameters to reproduce the data;

dispersed storage error encoding the data in accordance with second error coding dispersal storage function parameters to produce a plurality of sets of second encoded data slices; and

for each set of the plurality of sets of second encoded data slices, sending at least a write threshold number of second encoded data slices to a second set of DS units for storage therein.

2. The method of claim 1 further comprises:

creating an entry in a virtual dispersed storage network (DSN) address to physical location table to indicate an association between the plurality of sets of second encoded data slices and the second set of DS units; and

maintaining an entry in the virtual dispersed storage network (DSN) address to physical location table to indicate an association between the plurality of sets of encoded data slices and the set of DS units;

or

deleting the entry in the virtual dispersed storage network (DSN) address to physical location table to indicate an association between the plurality of sets of encoded data slices and the set of DS units.

3. The method of claim 1 further comprises one of:

sending a delete encoded data slice message to each DS unit of the first set of DS units, wherein the delete encoded data slice message includes a request to delete the encoded data slices for each set of the plurality of sets of encoded data slices;

when receiving a store data slice confirmation message indicating that the plurality of sets of second encoded data slices are stored in the second set of DS units, sending the delete encoded data slice message to each DS unit of the first set of DS units;

when a wait time period has elapsed after sending the at least some of the plurality of sets of second encoded data slices to a second set of DS units, sending the delete encoded data slice message to each DS unit of the first set of DS units; and

when a storage space required indicator is active, sending the delete encoded data slice message to each DS unit of the first set of DS units.

4. The method of claim 1, wherein each of the error coding dispersal storage function parameters and the second error coding dispersal storage function parameters comprises at least one of:

a pillars list;

a segmenting protocol;

a pre-slice data manipulation function;

a forward error correction encoding function;

a slicing pillar width;

a post-slice data manipulation function;

a write threshold; and

a read threshold.

5. The method of claim 1, wherein the determining data to migrate is based on one or more of:

an amount of data to move indicator;

a data transferred indicator;

a data transfer continuation indicator;

a DS managing unit message;

a DS unit query;

a DS unit message;

a newly allocated DS unit detection;

a location;

a schedule;

a list;

a predetermination;

an error message; and

a command.

6. A method comprises:

determining a pillar of encoded data slices to migrate, wherein the pillar of encoded data slices is stored in a first dispersed storage (DS) unit;

identifying a second DS unit;

retrieving at least some of the encoded data slices of the pillar of encoded data slices from the first DS unit; and

sending the at least some of the encoded data slices of the pillar of encoded data slices to the second DS unit for storage therein.

7. The method of claim 6 further comprises:

updating an entry in a virtual dispersed storage network (DSN) address to physical location table to indicate an association between the pillar of encoded data slices and the second DS unit; and

maintaining an entry in the virtual dispersed storage network (DSN) address to physical location table to indicate an association between the pillar of encoded data slices and the first DS unit;

or

deleting the entry in the virtual dispersed storage network (DSN) address to physical location table to indicate the association between the pillar of encoded data slices and the first DS unit.

8. The method of claim 6 further comprises one of:

sending a delete encoded data slice message to the first DS unit, wherein the delete encoded data slice message includes a request to delete the pillar of encoded data slices;

when receiving a store data slice confirmation message indicating that the pillar of encoded data slices are stored in the second DS unit set, sending the delete encoded data slice message to the first DS unit;

when a wait time period has elapsed after sending the at least some of the pillar of encoded data slices to the second DS unit, sending the delete encoded data slice message to the first DS unit; and

when a storage space required indicator is active, sending the delete encoded data slice message to the first DS unit.

9. The method of claim 6, wherein the determining the pillar of encoded data slices to migrate is based on one or more of:

an amount of encoded data slices to move indicator;

a data transferred indicator;

a data transfer continuation indicator;

a DS managing unit message;

a DS unit query;

a DS unit message;

a newly allocated DS unit detection;

a location;

a schedule;

a list;

a predetermination;

an error message; and

a command.

10. The method of claim 6, wherein the identifying the second DS unit is based on at least one of:

a DS managing unit message;

a DS unit query;

a DS unit message;

a newly allocated DS unit detection;

a location;

a DS unit utilization indicator;

a DS unit capacity indicator;

a schedule;

a predetermination;

an error message; and

a command.

11. A computer comprises:

an interface; and

a processing module operable to;

determine data to migrate, wherein the data is stored as a plurality of sets of encoded data slices in a first set of dispersed storage (DS) units;

retrieve, via the interface, at least a read threshold number of encoded data slices for each set of the plurality of sets of encoded data slices;

dispersed storage error decode the at least the read threshold number of encoded data slices for each set of the plurality of sets of encoded data slices in accordance with error coding dispersal storage function parameters to reproduce the data;

dispersed storage error encode the data in accordance with second error coding dispersal storage function parameters to produce a plurality of sets of second encoded data slices; and

send, via the interface, at least a write threshold number of second encoded data slices to a second set of DS units for storage therein for each set of the plurality of sets of second encoded data slices.

12. The computer of claim 11, wherein the processing module further functions to:

create an entry in a virtual dispersed storage network (DSN) address to physical location table to indicate an association between the plurality of sets of second encoded data slices and the second set of DS units; and

maintain an entry in the virtual dispersed storage network (DSN) address to physical location table to indicate an association between the plurality of sets of encoded data slices and the set of DS units;

or

delete the entry in the virtual dispersed storage network (DSN) address to physical location table to indicate an association between the plurality of sets of encoded data slices and the set of DS units.

13. The computer of claim 11, wherein the processing module further functions to perform one of:

send, via the interface, a delete encoded data slice message to each DS unit of the first set of DS units, wherein the delete encoded data slice message includes a request to delete the encoded data slices for each set of the plurality of sets of encoded data slices;

send, via the interface, the delete encoded data slice message to each DS unit of the first set of DS units when receiving, via the interface, a store data slice confirmation message indicating that the plurality of sets of second encoded data slices are stored in the second set of DS units;

send, via the interface, the delete encoded data slice message to each DS unit of the first set of DS units when a wait time period has elapsed after sending, via the interface, the at least some of the plurality of sets of second encoded data slices to a second set of DS units; and

send, via the interface, the delete encoded data slice message to each DS unit of the first set of DS units when a storage space required indicator is active.

14. The computer of claim 11, wherein each of the error coding dispersal storage function parameters and the second error coding dispersal storage function parameters comprises at least one of:

a pillars list;

a segmenting protocol;

a pre-slice data manipulation function;

a forward error correction encoding function;

a slicing pillar width;

a post-slice data manipulation function;

a write threshold; and

a read threshold.

15. The computer of claim 11, wherein the processing module functions to determine data to migrate based on one or more of:

an amount of data to move indicator;

a data transferred indicator;

a data transfer continuation indicator;

a DS managing unit message;

a DS unit query;

a DS unit message;

a newly allocated DS unit detection;

a location;

a schedule;

a list;

a predetermination;

an error message; and

a command.

16. A computer comprises:

an interface; and

a processing module operable to;

determine a pillar of encoded data slices to migrate, wherein the pillar of encoded data slices is stored in a first dispersed storage (DS) unit;

identify a second DS unit;

retrieve, via the interface, at least some of the encoded data slices of the pillar of encoded data slices from the first DS unit; and

send, via the interface, the at least some of the encoded data slices of the pillar of encoded data slices to the second DS unit for storage therein.

17. The computer of claim 16, wherein the processing module further functions to:

update an entry in a virtual dispersed storage network (DSN) address to physical location table to indicate an association between the pillar of encoded data slices and the second DS unit; and

maintain an entry in the virtual dispersed storage network (DSN) address to physical location table to indicate an association between the pillar of encoded data slices and the first DS unit;

or

delete the entry in the virtual dispersed storage network (DSN) address to physical location table to indicate the association between the pillar of encoded data slices and the first DS unit.

18. The computer of claim 16, wherein the processing module further functions to perform one of:

send, via the interface, a delete encoded data slice message to the first DS unit, wherein the delete encoded data slice message includes a request to delete the pillar of encoded data slices;

send, via the interface, the delete encoded data slice message to the first DS unit when receiving, via the interface, a store data slice confirmation message indicating that the pillar of encoded data slices are stored in the second DS unit set;

send, via the interface, the delete encoded data slice message to the first DS unit when a wait time period has elapsed after sending, via the interface, the at least some of the pillar of encoded data slices to the second DS unit; and

send, via the interface, the delete encoded data slice message to the first DS unit when a storage space required indicator is active.

19. The computer of claim 16, wherein the processing module functions to determine the pillar of encoded data slices to migrate based on one or more of:

an amount of encoded data slices to move indicator;

a data transferred indicator;

a data transfer continuation indicator;

a DS managing unit message;

a DS unit query;

a DS unit message;

a newly allocated DS unit detection;

a location;

a schedule;

a list;

a predetermination;

an error message; and

a command.

20. The computer of claim 16, wherein the processing module functions to identify the second DS unit based on at least one of:

a DS managing unit message;

a DS unit query;

a DS unit message;

a newly allocated DS unit detection;

a location;

a DS unit utilization indicator;

a DS unit capacity indicator;

a schedule;

a predetermination;

an error message; and

a command.