Method and system for data transfer optimization

• US 9,195,696 B2
• Filed: 06/11/2013
• Issued: 11/24/2015
• Est. Priority Date: 08/07/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Smith, Ian T.
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US13/914,709
• Publication Number:
  US 20140046917A1
• Time in Patent Office: 896 Days
• Field of Search: 707/696
• US Class Current: 1/1
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

A data migration system and method for migrating data objects from a source entity to a target entity, wherein the source entity includes infrastructure shared between discrete systems, the system including: a storage device storing an index of the data objects stored on the shared infrastructure and object attributes of the data objects, said index being in a normalized data model independent of native formats of the discrete systems; a selector adapted to select which objects to migrate based on at least one said object attribute; and an optimizer adapted to optimize the migration of data from the shared infrastructure to the target entity.

Claims

1. A data migration method for migrating data objects from a source entity to a target entity, wherein the source entity comprises an infrastructure shared between discrete systems, the method comprising:

providing a single index of the data objects stored on the shared infrastructure and object attributes of the data objects, the index being provided in a normalized format independent of native formats of the discrete systems;

selecting which of the data objects to migrate based on at least one object attribute;

splitting the migration of the selected data objects into a plurality of phases, each of the selected data objects is assigned to one of the plurality of phases, and the selected date objects of one phase are migrated at a different time than the selected data objects of another phase;

optimizing the migration of the selected data objects from the shared infrastructure to the target entity by grouping the selected data objects of each phase into migration sets based on the object attributes; and

migrating the selected data objects from the source entity to the target entity based on the migration phases established during splitting and the migration sets established during optimizing.

2. The data migration method according to claim 1, wherein the at least one object attribute used for selecting which data objects to migrate is one of object owner, group owner, data type, and expiration.

3. The data migration method according to claim 1, wherein the object attributes comprise at least one of customer data, site data, source data, node data, object data and fragment data.

4. The data migration method according to claim 1, wherein the shared infrastructure comprises storage media and the migration sets are based on the location of the selected data objects on the storage media, the relationship of the selected data objects to the respective discrete systems, and available access paths for the selected data objects from the discrete systems to the storage media.

5. The data migration method according to claim 4, wherein the selected data objects are ordered sequentially in each migration set based on their locations on the storage media.

6. The data migration method according to claim 1, wherein providing the single index of the data objects stored on the shared infrastructure and object attributes of the data objects comprises:

interrogating storage devices of the shared infrastructure using the standard command line interface particular to each respective storage devices;

extracting predetermined data from the storage devices; and

translating the extracted predetermined data into a normalized format.

7. The data migration method according to claim 1, wherein providing the single index of the data objects stored on the shared infrastructure and object attributes of the data objects comprises:

generating dump files which contain predetermined system configuration and state information in product specific formats by interrogating storage devices of the shared infrastructure;

extracting the predetermined system configuration and state information from the dump files; and

translating the extracted predetermined system configuration and state information from the product specific formats into a normalized format.

8. A data migration system for migrating data objects from a source entity to a target entity, wherein the source entity comprises an infrastructure shared between discrete systems, the system comprising:

one or more computer processors, one or more computer-readable storage media, and program instructions stored on one or more of the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising;

program instructions to store a single index of the data objects stored on the shared infrastructure and object attributes of the data objects, the index being in a normalized data model independent of native formats of the discrete systems;

program instructions to select which data objects to migrate based on at least one object attribute; and

program instructions to split the migration of the selected data objects into a plurality of phases, each phase has a different scheduled start time, each of the selected data objects is assigned to one of the plurality of phases, and the selected date objects of one phase are migrated at a different time than the selected data objects of another phase

program instructions to optimize the migration of the selected data objects from the shared infrastructure to the target entity by grouping the selected data objects of each phase into migration sets based on the object attributes.

9. The data migration system according to claim 8, wherein the at least one object attribute used to select which data objects to migrate is one of object owner, group owner, data type, and expiration.

10. The data migration system according to claim 8, wherein the object attributes comprise at least one of customer data, site data, source data, node data, object data and fragment data.

11. The data migration system according to claim 8, wherein the shared infrastructure comprises storage media and the migration sets are based on the location of the selected data objects on the storage media, the relationship of the selected data objects to the respective discrete systems, and available access paths for the selected data objects from the discrete systems to the storage media.

12. The data migration system according to claim 11, wherein the selected data objects are ordered sequentially in each migration set based on their locations on the storage media.

13. The data migration system according to claim 11, wherein the migration set allows migration of the selected data objects across parallel data paths.

14. A method comprising:

providing a single index of the data objects stored on the shared infrastructure and object attributes of the data objects, the index being provided in a normalized format independent of native formats of the discrete systems;

selecting which of the data objects to migrate based on at least one object attribute;

optimizing the migration of data from the shared infrastructure to the target entity;

splitting the migration of the selected data objects into a plurality of phases, each of the selected data objects is assigned to one of the plurality of phases, and the selected date objects of one phase are migrated at a different time than the selected data objects of another phase; and

optimizing the migration of the selected data objects from the shared infrastructure to the target entity by grouping the selected data objects of each phase into migration sets based on the object attributes.

15. The method according to claim 14, wherein the at least one object attribute used for selecting which objects to migrate is one of object owner, group owner, data type, and expiration.

16. The method according to claim 14, wherein the object attributes comprise at least one of customer data, site data, source data, node data, object data and fragment data.

17. The method according to claim 14, wherein the shared infrastructure comprises storage media and the migration sets are based on the location of the selected data objects on the storage media, the relationship of the selected data objects to the respective discrete systems, and available access paths for the selected data objects from the discrete systems to the storage media.

18. The method according to claim 14, wherein the selected data objects are ordered sequentially in each migration set based on their locations on the storage media.

19. The method according to claim 14, wherein providing the single index of the data objects stored on the shared infrastructure and object attributes of the data objects comprises:

interrogating storage devices of the shared infrastructure using the standard command line interface particular to each respective storage devices;

extracting predetermined data from the storage devices; and

translating the extracted predetermined data into a normalized format.

20. The method according to claim 14, wherein providing the single index of the data objects stored on the shared infrastructure and object attributes of the data objects comprises:

generating dump files which contain predetermined system configuration and state information in product specific formats by interrogating storage devices of the shared infrastructure;

extracting the predetermined system configuration and state information from the dump files; and

translating the extracted predetermined system configuration and state information from the product specific formats into a normalized format.