PARALLEL MIGRATION OF DATA OBJECTS TO CLUSTERED STORAGE

• US 20150178014A1
• Filed: 12/19/2013
• Published: 06/25/2015
• Est. Priority Date: 12/19/2013
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Nelson, Steven Boyd
• Granted Patent: US 9,207,873 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Data objects can be migrated, while live, to virtualized clustered storage arrays in an efficient manner to allow for efficient transition from non-clustered storage to the virtualized clustered storage arrays. A data migration specification indicates data objects to be migrated and parameters for the migration. The parameters include a source of a data object, a destination of the data object in the virtualized clustered storage arrays, and a transfer space. A migration engine validates and parses the data migration specification. For each unique association of source, destination, and transfer space, the migration engine instantiates a migration process that drives and monitors migration of the corresponding data object. The migration processes operate in parallel for migration of the specified data objects into the virtualized clustered storage arrays.

Claims

1. A method comprising:

determining a plurality of data objects to be migrated via a plurality of migration data paths, wherein each of the plurality of migration data paths indicates a source, a destination, and an intermediate storage and each of the plurality of migration data paths is unique among the plurality of migration data paths;

for each of the plurality of migration data paths, associating the migration data path and a corresponding one of the plurality of data objects with a different migration process;

each of the migration processes,

configuring a space in the intermediate storage indicated by the associated migration data path to receive data;

causing replication of the associated data object from the source indicated by the associated migration data path to the configured intermediate storage space;

causing replication of data in the configured intermediate storage space to the destination indicated by the associated migration data path.

2. The method of claim 1 further comprising:

each of the migration processes, establishing a data path between the source indicated by the associated migration data path and the configured intermediate storage space and establishing a data path between the configured intermediate storage space and the destination indicated by the associated migration data path.

3. The method of claim 1 further comprising:

each of the migration processes, updating migration state of the migration process.

4. The method of claim 1 further comprising:

validating data that indicates the plurality of data objects and the plurality of migration data paths to ensure uniqueness of each of the plurality of migration data paths among the plurality of migration data paths.

5. The method of claim 4, further comprising eliminating or disregarding a data object in the data that is associated with a migration data path that is not unique among the plurality of migration data paths.

6. The method of claim 4 further comprising delaying migration of a data object corresponding to a non-unique migration data path.

7. The method of claim 4 further comprising specifying a different intermediate storage in a non-unique migration data path.

8. The method of claim 1, wherein said configuring the space in the intermediate storage indicated by the associated migration data path to receive data is in accordance with a parameter associated with the data migration path.

9. The method of claim 1, wherein said associating the migration data path and a corresponding one of the plurality of data objects with a different migration process comprises instantiating each of the different migration processes.

10. A computer program product, the computer program product comprising:

a computer readable storage medium having computer usable program code embodied therewith, the computer usable program code comprising first program code and second program code, the first program code to,

determine a plurality of data objects to be migrated via a plurality of migration data paths, wherein each of the plurality of migration data paths indicates a source, a destination, and an intermediate storage and each of the plurality of migration data paths is unique among the plurality of migration data paths;

for each of the plurality of migration data paths, associate the migration data path and a corresponding one of the plurality of data objects with a different migration process, wherein each of the different migration processes is based on the second program code;

the second program code to,

determine a source, an intermediate storage, a destination, and a data object;

configure a space in the intermediate storage to receive data;

cause replication of the data object from the source to the configured intermediate storage space;

cause replication of data in the configured intermediate storage space to the destination.

11. The computer program product of claim 10, wherein the second program code further comprises program code to establish a data path between the source and the configured intermediate storage space and establish a data path between the configured intermediate storage space and the destination.

12. The computer program product of claim 10, wherein the second program code further comprises program code to updating migration state based on at least one of configuration of the intermediate storage space, replication of the data object from the source to the configured intermediate storage space, and replication of the data in the configured intermediate storage space to the destination.

13. The computer program product of claim 10, wherein the first program code further comprises program code to:

validate data that indicates the plurality of data objects and the plurality of migration data paths to ensure uniqueness of each of the plurality of migration data paths among the plurality of migration data paths.

14. The computer program product of claim 13, wherein the first program code further comprises program code to eliminate or disregard a data object in the data that is associated with a migration data path that is not unique among the plurality of migration data paths.

15. The computer program product of claim 13, wherein the first program code further comprises program code to delay migration of a data object corresponding to a non-unique migration data path.

16. The computer program product of claim 13, wherein the first program code further comprises program code to specify a different intermediate storage in a non-unique migration data path.

17. The computer program product of claim 10, wherein the first program code to, for each of the plurality of migration data paths, associate the migration data path and a corresponding one of the plurality of data objects with a different migration process comprises program code to instantiate the second program code for each of the plurality of migration data paths.

18. An apparatus comprising:

a processor;

a network interface; and

a machine-readable storage medium having a first computer usable program code and a second computer usable program code encoded therein,

the first computer usable program code executable by the processor to cause the apparatus to,

determine a plurality of data objects to be migrated via a plurality of migration data paths, wherein each of the plurality of migration data paths indicates a source, a destination, and an intermediate storage and each of the plurality of migration data paths is unique among the plurality of migration data paths;

for each of the plurality of migration data paths, associate the migration data path and a corresponding one of the plurality of data objects with a different migration process, wherein each of the different migration processes is based on the second program code;

the second computer usable program code executable by the processor to cause the apparatus to,

determine a source, an intermediate storage, a destination, and a data object;

configure a space in the intermediate storage to receive data;

cause replication of the data object from the source to the configured intermediate storage space;

cause replication of data in the configured intermediate storage space to the destination.

19. The apparatus of claim 18, wherein the first computer usable program code executable by the processor to cause the apparatus to, for each of the plurality of migration data paths, associate the migration data path and a corresponding one of the plurality of data objects with a different migration process comprises the first computer usable program code being executable by the processor to cause the apparatus to instantiate the second computer usable program code for each of the plurality of migration data paths.

20. The apparatus of claim 18, wherein the second computer usable program code further comprises computer usable program code executable by the processor to cause the apparatus to establish a data path between the source and the configured intermediate storage space and establish a data path between the configured intermediate storage space and the destination.