DATA MIGRATION SYSTEM

• US 20190324655A1
• Filed: 06/28/2019
• Published: 10/24/2019
• Est. Priority Date: 11/02/2017
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Nguyen, Minh, Ahuja, Neeraj
• Granted Patent: US 10,768,818 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Disclosed are some implementations of systems, apparatus, methods and computer program products for facilitating data migration. Data migration is customizable according to user-specified data formats. A data migration system monitors the data migration process and automatically retries data migration tasks that have failed. In addition, the results of data migration are reversed according to detected threshold conditions.

Claims

1. A system comprising:

a database system implemented using a server system, the database system configurable to cause;

generating data capable of being processed by a client device to cause display of a graphical user interface (GUI) configured to present, for each one of a plurality of source record types, for each one of a corresponding plurality of source fields, a plurality of user selectable options indicating a plurality of destination fields, each of the destination fields being associated with a corresponding one of one or more destination record types;

processing, for each of one or more of the source record types, for each of one or more of the corresponding source fields, a user selection of one of the plurality of user selectable options indicating a corresponding one of the destination fields;

responsive to processing the user selection for each of the one or more of the source fields for each of the one or more of the source record types, storing a field mapping that associates each of the one or more of the source fields, for each of the one or more of the source record types, with a corresponding one of the destination fields;

generating a plurality of destination data records using the field mapping;

storing the destination data records in the at least one database of the database system;

performing data migration of record data from at least a portion of the source data records to at least a portion of the destination data records according to the field mapping; and

responsive to detecting a threshold condition, reversing the data migration for the portion of the destination data records such that the record data stored in the portion of the destination data records is stored in the portion of the source data records.

2. The system as recited in claim 1, the database system further configurable to cause:

reversing the data migration for at least one of the portion of the destination data records by copying or transferring the record data from one of the destination fields of the destination data record to a corresponding source field of a corresponding one of the source data records.

3. The system as recited in claim 1, the database system further configurable to cause:

reversing the data migration for at least one of the portion of the destination data records by copying or transferring a pointer from one of the destination fields of the destination data record to a corresponding source field of a corresponding one of the source data records.

4. The system as recited in claim 1, the database system further configurable to cause:

updating, during data migration, a data structure such that each one of a plurality of bits indicates a data migration status of a corresponding one of a plurality of data migration tasks.

5. The system as recited in claim 1, the database system further configurable to cause detecting the threshold condition based at least, in part, on data migration statistics collected during the data migration, the data migration statistics including one or more of:

a data migration time indicating a time that has expired since the data migration was initiated;

a number of a plurality of data migration tasks that have failed to successfully complete;

a number of the data migration tasks that have successfully completed;

a percentage of the data migration tasks that have failed to successfully complete;

or

a percentage of the data migration tasks that have successfully completed.

6. The system as recited in claim 1, the database system further configurable to cause:

detecting the threshold condition by applying a set of rules in real-time during the data migration.

7. The system as recited in claim 1, the database system further configurable to cause:

monitoring a data structure in real-time during the data migration to determine whether any data migration tasks have failed.

8. A computer program product comprising computer-readable program code capable of being executed by one or more processors when retrieved from a non-transitory computer-readable medium, the program code comprising computer-readable instructions configurable to cause:

generating data capable of being processed by a client device to cause display of a graphical user interface (GUI) configured to present, for each one of a plurality of source record types, for each one of a corresponding plurality of source fields, a plurality of user selectable options indicating a plurality of destination fields, each of the destination fields being associated with a corresponding one of one or more destination record types;

processing, for each of one or more of the source record types, for each of one or more of the corresponding source fields, a user selection of one of the plurality of user selectable options indicating a corresponding one of the destination fields;

responsive to processing the user selection for each of the one or more of the source fields for each of the one or more of the source record types, storing a field mapping that associates each of the one or more of the source fields, for each of the one or more of the source record types, with a corresponding one of the destination fields;

generating a plurality of destination data records using the field mapping;

storing the destination data records in the at least one database of the database system;

performing data migration of record data from at least a portion of the source data records to at least a portion of the destination data records according to the field mapping; and

responsive to detecting a threshold condition, reversing the data migration for the portion of the destination data records such that the record data stored in the portion of the destination data records is stored in the portion of the source data records.

9. The computer program product as recited in claim 8, the program code comprising instructions further configured to cause:

reversing the data migration for at least one of the portion of the destination data records by copying or transferring the record data from one of the destination fields of the destination data record to a corresponding source field of a corresponding one of the source data records.

10. The computer program product as recited in claim 8, the program code comprising instructions further configured to cause:

reversing the data migration for at least one of the portion of the destination data records by copying or transferring a pointer from one of the destination fields of the destination data record to a corresponding source field of a corresponding one of the source data records.

11. The computer program product as recited in claim 8, the program code comprising instructions further configured to cause:

updating, during data migration, a data structure such that each one of a plurality of bits indicates a data migration status of a corresponding one of a plurality of data migration tasks.

12. The computer program product as recited in claim 8, the program code comprising instructions further configured to cause:

detecting the threshold condition based at least, in part, on data migration statistics collected during the data migration, the data migration statistics including one or more of;

a data migration time indicating a time that has expired since the data migration was initiated;

a number of a plurality of data migration tasks that have failed to successfully complete;

a number of the data migration tasks that have successfully completed;

a percentage of the data migration tasks that have failed to successfully complete;

or

a percentage of the data migration tasks that have successfully completed.

13. The computer program product as recited in claim 8, the program code comprising instructions further configured to cause:

detecting the threshold condition by applying a set of rules in real-time during the data migration

14. The computer program product as recited in claim 8, the program code comprising instructions further configured to cause:

monitoring a data structure in real-time during the data migration to determine whether any data migration tasks have failed.

15. A method, comprising:

generating data capable of being processed by a client device to cause display of a graphical user interface (GUI) configured to present, for each one of a plurality of source record types, for each one of a corresponding plurality of source fields, a plurality of user selectable options indicating a plurality of destination fields, each of the destination fields being associated with a corresponding one of one or more destination record types;

processing, for each of one or more of the source record types, for each of one or more of the corresponding source fields, a user selection of one of the plurality of user selectable options indicating a corresponding one of the destination fields;

responsive to processing the user selection for each of the one or more of the source fields for each of the one or more of the source record types, storing a field mapping that associates each of the one or more of the source fields, for each of the one or more of the source record types, with a corresponding one of the destination fields;

generating a plurality of destination data records using the field mapping;

storing the destination data records in the at least one database of the database system;

performing data migration of record data from at least a portion of the source data records to at least a portion of the destination data records according to the field mapping; and

responsive to detecting a threshold condition, reversing the data migration for the portion of the destination data records such that the record data stored in the portion of the destination data records is stored in the portion of the source data records.

16. The method as recited in claim 15, further comprising:

reversing the data migration for at least one of the portion of the destination data records by copying or transferring the record data from one of the destination fields of the destination data record to a corresponding source field of a corresponding one of the source data records.

17. The method as recited in claim 15, further comprising:

reversing the data migration for at least one of the portion of the destination data records by copying or transferring a pointer from one of the destination fields of the destination data record to a corresponding source field of a corresponding one of the source data records.

18. The method as recited in claim 15, further comprising:

updating, during data migration, a data structure such that each one of a plurality of bits indicates a data migration status of a corresponding one of a plurality of data migration tasks.

19. The method as recited in claim 15, further comprising:

detecting the threshold condition based at least, in part, on data migration statistics collected during the data migration, the data migration statistics including one or more of;

a data migration time indicating a time that has expired since the data migration was initiated;

a number of a plurality of data migration tasks that have failed to successfully complete;

a number of the data migration tasks that have successfully completed;

a percentage of the data migration tasks that have failed to successfully complete;

or

a percentage of the data migration tasks that have successfully completed.

20. The method as recited in claim 15, further comprising:

detecting the threshold condition by applying a set of rules in real-time during the data migration.